Sonification

Sonification is the technical term for embedding information into artificially generated sounds.
In our project we want to explore how spatial information can be communicated to users via sonification. The goal is to map a spatial relation between two points. To describe this relation between 'start' and 'destination' in three-dimensional space, six half-axes are needed. For each of these semi-axes a unique tone must be used to describe it.

The sonification developed by Tim Ziemer has also set itself the goal of permanently conveying the information of direction and distance. It was developed on the basis of findings in psychoacoustics and is thus intended to enable blind navigation. In the long term, information could also be conveyed to surgeons (intra-operative information).

The model was designed and developed in PureData initially as a 2D model, which was extended and refined to a 3D system in the bachelor project. This existing system has now been transferred to the programming language C# and the game engine Unity to increase compatibility with different operating systems. Side effects are an increased maintainability and stability as well as an easier barrier to entry for textual programmers.

Cognitive Training

Cognitive training describes within this project the training of spatial imagination and orientation, as well as the coordination between hand and eyes. The aim of this training is to improve the control of tools, despite discrepancies that this control shows.

Controlling tools in three-dimensional space becomes difficult if the tools themselves can only be seen on a monitor and not in 3D. In addition, movement and vision can be decoupled by possible rotation, magnification and displacement between monitor and tool.

Similar problems are faced by surgeons during laparoscopic (minimally invasive) procedures, which are a challenge especially for newcomers in this field.
The aim is therefore to train the skills required for such interventions detached from the actual subject matter. Tina Vajsbaher is investigating the influence of these skills on the learning of minimally invasive surgery (Spatial Cognition in Surgical Practice).
But also players outside the medical environment are given the opportunity to train these skills.

Adaptivity

To be able to assess the learning progress and provide the user with tasks, which correspond to their current abilities as well as possible, but are nevertheless demanding, the goal is to create an adaptive system.

To do this, the appropriate skills must first be identified, and then tasks or games are developed that train these skills whereby a conclusion to the learning progress can be made.

The goal is to find or generate a level of difficulty that suits the user after only a few games. Generated levels also lead to a greater variety within the game.

The Game
Overview
CURAT Logo with Game

The Project

Our games playfully improve spatial thinking, both visually and audibly.

group photo CURAT

The Team

This project is the product of collective work of different groups, such as gamification, adaptivity and controls.