01/2 3D reconstruction from endoscopic video sequences

		Fig. 1: Navigation and referencing using endoscopes Fig. 2: Model update

Real-time quantitative measurements and 3D visualisation during endoscopic surgery can provide the clinician with valuable additional information. The goal is to use the endoscope rather as imaging device than just as a keyhole!

Providing this additional information without adding another tool to the operating scene and without demanding more training for the surgeon is another important aspect (less cost, less risk, shorter surgery).

In a first step we want to work on scenes where something can be tracked rigidly such as in orthopedic or even neurosurgical surgery. In a second phase we might consider to bring this technology to more complex scenes where deformations can occur. But we will restrict ourselves to the case where this motion and deformation can be parameterised.

Once the relationship between the target anatomy, the tools and the preoperative model is established, it is also possible to create a see-through patient with Augmented Reality techniques so the surgeon has all information in the same display.

This project is a joint project between the Computer Vision Lab at the ETH Zurich and the Virtual Reality and Active Interfaces Group at the EPF Lausanne.

Navigation and referencing using endoscopes (Fig. 1)

The target anatomy can be tracked optically using the endoscope, which is again tracked by the same system as the surgeon's tools. As the transformation bewteen camera and tracking system is known, the target is also tracked globally. This allows to navigate easily inside the human body without direct sight on the target. The whole system can be calibrated within a few minutes to reach best possible accuracy for each surgery.

Model update (Fig. 2)

If the pre-operative model is available in advance (such as CT/MRI), the model built from the tracked endoscope can be per-operatively updated and improved and also helps to better initialize the system.

Setup

The setup consists of a distortion free endoscope mounted on a digital camera (firewire) which is tracked by the optical tracking system (Atracsys) giving a highly accurate idea of the camera's pose. After a calibration step using Hand-Eye calibration techniques we know the relationship between the camera's image plane and the tracked marker attached to the camera which allows us to compute precise 3D reconstruction. The surgeon's tools are equally tracked by the tracking system and therefore all necessary transformations between all involved parts are known allowing for navigation and referencing.
A complete software package is already finished, allowing to see the real scene as a 3D model in a virtual world and at the same time the view through the endoscope and processed information.