Development and Evaluation of a Haptic-visual Learning System for Surgical Operations

The Challenge

Surgeons sometimes have to perform very complex operations on bony tissue, one example being surgical cochlear implants. It is important in such operations to avoid injuring delicate structures, such as nerves and blood vessels, and to ensure that the surgery does as little damage as possible. Before surgeons are able to carry out such an operation safely and without making any mistakes, they have to undergo comprehensive theoretical and practical training – some of it on the job – and need many years of experience.

In this context, Fraunhofer IIS has been working on the haptic (tactile) modeling of tissue, the segmentation and visualization of ear structures, and the evaluation of the demonstrator to be developed.

Our Contribution

Varying osseous transparency and visibility of risk structures in the middle ear
© Fraunhofer IIS
Varying osseous transparency and visibility of risk structures in the middle ear. The nerves (chorda tympani, nervus facialis) are shown in yellow, while the cochlea and semicircular canals (critical to sense of balance) are shown in gray-blue. From left to right, the transparency decreases and the difficulty increases.

We are supporting the further training of surgeons by developing the “HaptiVisT“ system, an haptic-visual training system for cochlear implants. To this end, real clinical CT data from the inner and middle ear are interactively segmented (“CI-Wizard“), and the structures modeled there (nerves, cochlea, ossicles, etc.) are used for an interactive surgical training system.

“HaptiVisT“ is structured as a so-called serious game with hidden learning. The results-oriented evaluation of the demonstrator´s fitness for purpose is carried out in parallel to the development process in order to uncover possible technical or didactic errors before completion of the system. This assessment involves three time-staggered evaluations, which focus on surgical, didactic, and haptic-ergonomic acceptance criteria.


The continuing further education of medical staff in the field of minimally invasive surgery is not restricted to a few specialist centers, but can also be extended to smaller hospitals in more rural areas. Expanding practice-oriented education and further training in specialized surgical disciplines can go a long way toward remedying the lack of specialist doctors across the board.

More Information


Your Development Partner for Medical Image Analysis

Talk to us – together we will find a solution for your medical image processing challenge:  -> Please send an email if you are interested

Software based Image Analysis and Enhancement

Our focus are the domains of Microscopy and Endoscopy.