Projekte
Fraunhofer Institute for Integrated Circuits
The Whole’O’Hand-System
Workflow
The main workflow of the Whole’O’Hand system starts with the pre-operative surgical planning based on volumetric CT data. The extracted planning data is then sent to the navigation control system, which is part of the surgical control unit (SCU). Based on the SCU which is located at the operation table, all sub-systems can be monitored, coordinated and controlled. In order to register the patient to the coordinate system of the SCU, a 3D stripe projector is used, whose acquired surface grid is registered to the abdomen surface extracted from the CT data. Based on this registration as well as on further surgical planning data, the positions of the trocars on the surface of the patient’s abdomen are located and marked via light projection. Hence, an optimal pre-operative planned access can be set for the complete intervention, which furthermore relates to the working volume of the instrument carrier system.
Next, the two instrument carrier systems (ICS) are moved towards the operation table. The ICS supports the surgeon by hosting and guiding the instruments. Furthermore, an innovative revolver-based instrument exchange system (IES) allows the direct change of instruments and sensors without instrument withdrawal inside the patient’s abdomen, which reduces the time needed to remove and exchange the instrument through the trocar.
After positioning of the trocars, a 3D-ultrasound sensor is inserted to intra-operatively scan the liver surface and thus reconstruct the liver volume. The liver volume is then registered to the pre-operative planning data from the CT, thus transforming the planning data into the actual intra-operative (and possible by air inflation deformed) coordinate system
During the resection the liver can be re-scanned permanently by means of the ultrasound sensor, thus allowing a constant matching between the planning data and the actual site. Furthermore the resection boundaries can continuously be controlled by the ultrasound device, thus allowing the detection of any dangerous changes. Possible accidents can be avoided by so-called active constraints-mechanisms. If necessary, the planning data can be adapted accordingly based on the constantly acquired volumetric ultrasound data.
Intra-Operative Surface Scanning
Within the pre-operative planning phase, the working area within the patient will be set. This information can be used to further plan the incision points for the trocars. This is an important step as the ICS only allows a constricted working volume. To fully exploit this volume, an ideal positioning between the patient and the ICS is needed. As mentioned above an optical 3D-scanner is used to scan the surface of the patient on the table and relate these surfaces to the planning data. After the insufflation, the access for the two ICS’s are set in such a way, that a direct access to the organ is possible. A guiding support assists the surgeon during the orientation of the ICS to yield the optimal position.
Navigation and Registration
The vessel trees of the liver are extracted and represented symbolically as a directed graph from pre-operatively obtained CT-scans of the abdomen as well as intraoperatively acquired 3D-ultrasound image. To generate such a symbolical description of the liver vessel system, the 3D CT/US image volumes are manipulated using a distinct chain of image processing methods. The first step yields a segmentation of the vessel tree, which is transformed into a so-called graph-based skeleton, representing the center lines of the vessels. Finally the skeleton is converted into a graph-based symbolic description. Based on this representation, a registration of pre- and intra-operative image data is possible.
For the actual registration of the pre-operative CT and the intra-operative 3D-ultrasound data, most of the correlating bifurcations between vessel trees are automatically detected and can be interactively refined to correct wrong matches. Based on this representative set of correlated landmarks between both image modalities, the deformation parameters can be obtained. Using this registration and the related deformation and transformation data, pre-operative computed resection-plans as well as resection boundaries can be transferred to the interventional site and be used and visualized intra-operatively together with the continuously acquired 3D-ultrasound volumes. Hence an integrated visual control of the resection and the resection boundaries is possible. Furthermore, the current positions of the instruments can be augmented to the fused images.
Instrument Carrier System
Two robotic manipulators serve as instrument carriers in the Whole’O’Hand concept. In our first setup two indus-trial robots are integrated into the oper-ating room. They are either installed on mobile platforms or mounted beneath the ceiling in order not to interfere with the surgical workflow or the clinical staff. Each of the manipulators carries one instrument exchange system. A six degree of freedom control interface is directly mounted to the instrument carrier which gives the surgeon the possibility to steer the instruments in an intui-tive, fast and precise way. The instrument car-riers are able to communicate with the surgical control unit (SCU). When the surgeon resects the tumor the instrument position is permanently compared to the planned and up-dated resection corridors. If the surgeon intends to leave these planned zones the instrument carriers save him from injuring sensitive regions like blood vessels by applying brake forces to the manipulators’ axes (active constraints). Beyond the scope of the project the carriers could in future become able to perform (semi-) automated procedures in coordination with new instruments, as e.g. suturing.
Instrument Exchange System
The objective of the Whole’O’Hand instrument exchange system (IES) are simplification of use and the exchange fast of laparoscopic instruments is. In standard laparoscopic procedures, instrument changes account for a significant amount of time; they interrupt the surgeon’s concen-tration phase and involve a reorientation of the new instrument at the region of interest. Within the Whole’O’ Hand instrument exchange system newly developed capsular instruments (see Fig. 4) are stored outside the body in a cylindrical magazine and are automatically positioned at situ via a hollow cylindrical shaft using a hydraulic mechanism. At the proximal end of the shaft inside the patient’s abdomen the capsular instruments are electrically contacted in order to establish electrosurgical functionalities or allow electrical feedback from sensors. Up to six different graspers, forceps, HF-instruments or tip-chip endoscopic cameras can be stocked and automatically exchanged within a few seconds.
3D-Sonography for Navigation and Resections Guidance
Navigation as well as intra-operatively localizing tumors a sweeping sonography system is used, which produces three-dimensional images of the liver. These volumes are digitally processed and registered to the pre-operatively acquired planning data. Thus, the system provides visualization for the resection process to the surgeon and gives notice of a possible constraint violation.
Placed at a single position, the sonography system samples several cross sections by means of a sweeping mechanism. These registered images are interpolated to a small volume part. During an initial scan across the liver surface a set of overlapping ultrasound volumes is acquired and subsequently merged to a coherent volume of the entire organ. During resection the ultrasound volume is dynamically updated by frequent rescans of the area of surgical interest.
While the sweeping mechanism provides pose information of each of the cross sections to map them into the polume, the instrument carrier system delivers necessary position information for volume fusion. As sophisticated approach a miniaturized video sensor is laterally attached to the ultrasound probe to provide precise position information in case of organ motion. By processing the video stream, the relative motion between instrument and organ is detected and used to support the positioning of the resection instruments.
Based on an automatic or interactive segmentation of the vessel tree, the navigation system obtains a registration of the ultrasound data with the pre-operative CT planning data. Hence, the planning data can be transferred to the cur-rent operative site. Thus, the surgeon can be provided with an augmented visualization of relevant constraints and ad-ditional information. By means of frequent rescans the current surgical activities are monitored and guidance is provided to assure compliance with transaction constraints.
Multimodal Diagnostic Imaging
The correct diagnosis of lesions usually requires a conventional biopsy followed by a histological analysis. To reduce the related costs, improve the accuracy whilst reducing the number of biopsies taken in a specific area and to acelerate the process of diagnosis by analyzing the area of interest right during the intervention, an approach for image-based computer-assisted diagnostis (CAD) is developed. The most important characteristics are the multimodal image acquisition and the semi-automatic online image processing.
Using a high-resolution endoscopic camera with a multi-modal (autofluorescent and whitelight) illumination, char-acteristical tissue features of the lesions like textual, mor-phological, colorimetric and functional parameters can be calculated. By means of a content-based image-retrieval approach, images with similar characteristics can be selected from a histologically validated reference data base. They can be used for early detection, a more confident diagnosis and to clarify the necessity of a biopsy. Furthermore, the search for similar reference images and the presentation of similar cases can support the approach of case-based reasoning.
The proposed system can be integrated in the established diagnostic workflow where the acquired images are pre-sented on a high-resolution monitor. If lesions are depicted on the monitor, a snapshot of the current display can be taken by using a foot-switch. Then the questionable lesion can be delineated interactively by using a touch-screen Based on the segmented region of interest the sys-tem presents the most similar reference images and pro-vides a suggestion for the diagnosis, which can then be used for treatment planning.
Set Bookmark