In order to support gastroscopic examinations and enhance the documentation possibilities of the esophagus, Fraunhofer Institute IIS has developed a novel approach providing panoramic images as well as a 3D reconstruction of the esophagus, based on the digital endoscopic video data.
Using endoscopic video sequences of the esophagus during a withdrawal from the stomach with constant speed as input data, s. Fig. 1a, the “TubeStitcher“ software converts these videos frame-by-frame into a so-called panoramic map of the esophagus wall, similar to cutting a paper roll open and flattening it.
In this “video map“ each row relates to a set of pixels on the border of the esophagus at one point of time. Hence, as the endoscope is withdrawn through the esophagus, in each frame a new set of pixels on the esophagus wall is taken and converted to a new line in the “video map“. As the endoscope is withdrawn from the stomach upwards, the rows on the “video map“ are arranged accordingly: rows on the bottom relate to frames on the lower end of the esophagus, whereas rows on the top relate to frames near the larynx, s. Fig. 1b.
Once such a “video map“ (or “esophagram“) has been obtained, it can be used to view the esophagus wall in a different way, as it is now flattened and can be checked and »read like a map« in a very short time.
Specifically, lesions depicted on this “esophagram“ can then interactively be marked or tagged, hence showing their appearance in the original and unmodified video data in a new window as a cross-reference, s. Fig. 2. This feature allows a smooth transition between the “esophagram“ or “video map“ and the original video image, since the rows in the “esophagram“ are digitally linked to their contributing set of pixels and reference frames in the endoscopic video.
Finally, based on the “video map“ or “esophagram“ a 3D reconstruction of the tubular esophagus wall can be approximated, s. Fig. 3. This 3D reconstruction, which depicts the texture of the examined esophagus can then interactively be moved and examined, hence yielding complete new interactive viewing and assessment possibilities of the upper GI tract.