HandsOn.surgery - Virtual Immersive Surgical Simulator


HandsOn.surgery is a surgical simulator that addresses all senses. The training is carried out with individualized 3D models from patients segmented from real volumetric image data (e.g. CT, DVT, MRI).


Press release

See, feel, train

The prototype of the virtual surgery trainer “HandsOn.surgery” helps surgeons prepare for individual patient cases prior to surgery, and enables them to practice the surgery.

Video (german)

See, Feel, Hear – Virtual Immersive Surgical Simulator

The virtual surgery trainer HandsOn.surgery was developed in the HaptiVisT project.



  • 3D patient model data extracted from volumetric data displayed on auto-stereoscopic 3D display (no glasses required)
  • A rotated mirror achieves a realistic distance between surgeon and surgical site and emulates the view through a surgical microscope
  • Tools selection (milling device with diamond or blades bit) and controls displayed on a 2D touchscreen


  • Realistic drilling sounds captured during a surgery


  • Original surgical milling hand piece controlled by haptic arm with force feedback, vibration and foot pedal
  • Motion-captured 3D-printed aspirator


Auto-stereoscopic 3D display, tilted mirror, 2D touchscreen, haptic arm, 3D-printed aspirator, hand motion-capture device, foot pedal, PC 

Scenario: Cochlear Implantation

The objective is to drill a hole into the temporal bone behind the ear to access the hearing channel and make space for the cochlear implant (CI) and its electrodes. It is crucial that the surgeon stays clear of the facial nerve and other sensitive structures. The inner and middle ear structures (cochlea, nerves, ossicles, vestibularis) have been segmented from a CT/DVT scan and are highlighted in colors in the virtual 3D model.

Advantage of Virtual Training

Low Running Costs
A virtual simulator has very low running costs compared to one-time usable 3D-printed models, human corpses or animal cadavers

Objective Evaluation
Each session is objectively and quantitatively evaluated. This way
        the trainee’s learning curve can be tracked over time
        the trainee’s surgical result can be compared against a golden standard created by an experienced expert

Individualizable Set of Training Cases 
Training cases of varying difficulty levels, different pathologies and different patients can be loaded

High Availability 

The virtual trainer is always available and does not require lengthy preparations beforehand or cleaning up afterwards. It does not require a lab or surgical theater




Orthopedic surgery/traumatology

  • Hip replacement
  • Shoulder and elbow surgery
  • Total joint reconstruction (arthroplasty)
  • Pediatric orthopedics
  • Spine surgery
  • Surgical sports medicine
  • Orthopedic trauma

Ear Nose Throat (ENT) surgery

  • Cochlear implantation
  • Mastoidectomy
  • Endocrine surgery
  • Sinus surgery

Oral and maxillofacial surgery (OMS)

  • Cranio-maxillofacial trauma
  • Craniofacial surgery
  • Pediatric maxillofacial surgery
  • Cleft surgery
  • Dental applications

Possible future extensions

  • Drilling-Simulation
  • Interaction with soft tissue

Core Technology

The core technology of HandsOn.surgery is the highly realistically modeled interaction between the tool (milling device) and the personalized patient bone model. The underlying physics have been handcrafted by an interdisciplinary team of Fraunhofer IIS scientists and an experienced Ear-Nose-Throat (ENT) specialist. Four other ENT-specialists from multiple hospitals have subsequently evaluated and validated the physical simulation.



HandsOn.surgery is not certified for diagnostics or therapy of diseases. It is not a certified medical product.

More Information


Your Development Partner for Medical Image Analysis

Talk to us – together we will find a solution for your medical image processing challenge:  -> Please send an email if you are interested

Software based Image Analysis and Enhancement

Our focus are the domains of Microscopy and Endoscopy.