HandsOn.surgery - Virtual Immersive Surgical Simulator

See.Feel.Train.

HandsOn.surgery is a surgical simulator that addresses all senses. The training is carried out with individualized 3D models from patients segmented from real volumetric image data (e.g. CT, DVT, MRI).

 

Press release

See, feel, train

The prototype of the virtual surgery trainer “HandsOn.surgery” helps surgeons prepare for individual patient cases prior to surgery, and enables them to practice the surgery.

Video (german)

See, Feel, Hear – Virtual Immersive Surgical Simulator

The virtual surgery trainer HandsOn.surgery was developed in the HaptiVisT project.

Features

See

  • 3D patient model data extracted from volumetric data displayed on auto-stereoscopic 3D display (no glasses required)
  • A rotated mirror achieves a realistic distance between surgeon and surgical site and emulates the view through a surgical microscope
  • Tools selection (milling device with diamond or blades bit) and controls displayed on a 2D touchscreen

Hear

  • Realistic drilling sounds captured during a surgery

Feel

  • Original surgical milling hand piece controlled by haptic arm with force feedback, vibration and foot pedal
  • Motion-captured 3D-printed aspirator

Components:

Auto-stereoscopic 3D display, tilted mirror, 2D touchscreen, haptic arm, 3D-printed aspirator, hand motion-capture device, foot pedal, PC 

Scenario: Cochlear Implantation

The objective is to drill a hole into the temporal bone behind the ear to access the hearing channel and make space for the cochlear implant (CI) and its electrodes. It is crucial that the surgeon stays clear of the facial nerve and other sensitive structures. The inner and middle ear structures (cochlea, nerves, ossicles, vestibularis) have been segmented from a CT/DVT scan and are highlighted in colors in the virtual 3D model.

Advantage of Virtual Training

Low Running Costs
A virtual simulator has very low running costs compared to one-time usable 3D-printed models, human corpses or animal cadavers


Objective Evaluation
Each session is objectively and quantitatively evaluated. This way
        the trainee’s learning curve can be tracked over time
        the trainee’s surgical result can be compared against a golden standard created by an experienced expert


Individualizable Set of Training Cases 
Training cases of varying difficulty levels, different pathologies and different patients can be loaded


High Availability 

The virtual trainer is always available and does not require lengthy preparations beforehand or cleaning up afterwards. It does not require a lab or surgical theater

 

 

Applications

Orthopedic surgery/traumatology

  • Hip replacement
  • Shoulder and elbow surgery
  • Total joint reconstruction (arthroplasty)
  • Pediatric orthopedics
  • Spine surgery
  • Surgical sports medicine
  • Orthopedic trauma

Ear Nose Throat (ENT) surgery

  • Cochlear implantation
  • Mastoidectomy
  • Endocrine surgery
  • Sinus surgery

Oral and maxillofacial surgery (OMS)

  • Cranio-maxillofacial trauma
  • Craniofacial surgery
  • Pediatric maxillofacial surgery
  • Cleft surgery
  • Dental applications

Possible future extensions

  • Drilling-Simulation
  • Interaction with soft tissue

Core Technology

The core technology of HandsOn.surgery is the highly realistically modeled interaction between the tool (milling device) and the personalized patient bone model. The underlying physics have been handcrafted by an interdisciplinary team of Fraunhofer IIS scientists and an experienced Ear-Nose-Throat (ENT) specialist. Four other ENT-specialists from multiple hospitals have subsequently evaluated and validated the physical simulation.

 

Note

HandsOn.surgery is not certified for diagnostics or therapy of diseases. It is not a certified medical product.

More Information

Brochure

Engineering. Imaging. Sensors. Analysis.

Download Section

Each document, such as application note, brochure or others, and each video, can be found in our download section.

Brochure “Medical Image Processing“

Development partner for personalized medical engineering

Research and development services from a single source