TubeStitcher

A novel approach providing panoramic images as well as a 3D reconstruction of the esophagus, based on the digital endoscopic video data, to support gastroscopic examinations and enhance the documentation possibilities of the esophagus.

Panoramic Endoscopy of the Esophagus

The Challenge

For clinical assessment and the examination of the upper gastrointestinal (GI) tract, consisting of stomach and esophagus, video-endoscopic imaging is applied. For documentation of these procedures, usually representative still images as well as video clips are used. Nevertheless, due to the tight tubular geometry of the esophagus on one hand and the compact design of endoscopic devices on the other, the field of view in the esophagus is considerably constrained. Furthermore, contextual information of the field of view is limited and inhomogeneous illumination interferes with the examination.

Our Solution: TubeStitcher

TubeStitcher - Panorama-Dastellung der Speiseröhre
© Photo Fraunhofer IIS

Fig. 1a: Endoscopic image of the esophagus with marked details, Fig. 1b: Corresponding panorama image with preserved details, Fig. 1c: Enlarged panorama image

TubeStitcher - Panorama-Darstellung der Speiseröhre
© Photo Fraunhofer IIS

Fig. 2: Panorama image of an esophagus with marked details (center), corresponding original frames with mapping of the marked structures (left and right)

In order to support gastroscopic examinations and enhance the documentation possibilities of the esophagus, Fraunhofer Institute IIS has developed a novel approach providing panoramic images as well as a 3D reconstruction of the esophagus, based on the digital endoscopic video data.

Using endoscopic video sequences of the esophagus during a withdrawal from the stomach with constant speed as input data, s. Fig. 1a, the “TubeStitcher“ software converts these videos frame-by-frame into a so-called panoramic map of the esophagus wall, similar to cutting a paper roll open and flattening it.

In this “video map“ each row relates to a set of pixels on the border of the esophagus at one point of time. Hence, as the endoscope is withdrawn through the esophagus, in each frame a new set of pixels on the esophagus wall is taken and converted to a new line in the “video map“. As the endoscope is withdrawn from the stomach upwards, the rows on the “video map“ are arranged accordingly: rows on the bottom relate to frames on the lower end of the esophagus, whereas rows on the top relate to frames near the larynx, s. Fig. 1b.

Once such a “video map“ (or “esophagram“) has been obtained, it can be used to view the esophagus wall in a different way, as it is now flattened and can be checked and »read like a map« in a very short time.

Specifically, lesions depicted on this “esophagram“ can then interactively be marked or tagged, hence showing their appearance in the original and unmodified video data in a new window as a cross-reference, s. Fig. 2. This feature allows a smooth transition between the “esophagram“ or “video map“ and the original video image, since the rows in the “esophagram“ are digitally linked to their contributing set of pixels and reference frames in the endoscopic video.

Finally, based on the “video map“ or “esophagram“ a 3D reconstruction of the tubular esophagus wall can be approximated, s. Fig. 3. This 3D reconstruction, which depicts the texture of the examined esophagus can then interactively be moved and examined, hence yielding complete new interactive viewing and assessment possibilities of the upper GI tract.

Endoscopic Video of the Esophagus with TubeStitcher

Benefits

TubeStitcher - Panorama-Darstellung der Speiseröhre
© Photo Fraunhofer IIS

Fig. 3: 3D Reconstruction of an Esophagus

The “TubeStitcher“ software converts esophageal endoscopic video data into an intuitively assessable “esophagram“ for enhanced digital documentation in gastroscopy. These “esophagram“ can provide the missing context information and can be used to examine the complete esophagus at a glance. Using digital linking mechanisms, regions of interest in the “esophagram“ can be interactively tagged and provide the same regions in the original endoscopic video.

 

 

Thus the “TubeStitcher“ provides:

  • enhanced documentation of the esophagus,
  • improved context-aware diagnosis,
  • interactive assessment of depicted lesions in a multi-view approach and
  • new educational possibilities.

Furthermore, the 3-dimensional reconstruction offers a condensed representation of the esophagus geometry, potentially highlighting deformations or perforations. Also, the “TubeStitcher“ can conceptually be applied to video-sequences of other tubular structures such as the urethra, or the trachea.

More Information

Brochure “Medical Image Processing“

Download Section

Each document, such as application note, brochure or others, and each video, can be found in our download section.