Augmented Reality (AR) based orthopaedic simulation

		Camera and marker mounted on the helmet. HMD for displaying the virtual object. Image displayed with the HMD to the user. Virtual teapot added to the scene.

Central objective of this research line is to determine the value of Augmented Reality (AR) techniques in the field of surgical simulation - the driving application being a bone repositioning training setup. Currently, two main threads are followed.

The first part focuses on the AR elements of the training system. The setup consists of a firewire camera for video capture, a HMD for display as well as an infra-red 6 DoF tracking system which reports orientation and position of an emitter mounted on the camera. The purpose of this device is to track the user's head position while the AR system is running, since the challenge in AR is to maintain an accurate alignment between the real and the virtual world. In order to achieve this goal, the camera pose relative to the working environment has to be calculated as accurately as possible. Therefore, we have developed a new method to accurately estimate the transformation between the camera and the emitter. Based on the observation that some color cameras are sensitive to infrared wavelengths, we have designed a calibration box consisting of infrared LEDs detected simultaneously by a commercial tracking system (NDI's Optotrak) and the color camera. This calibration object allows us to accurately determine the relationship between camera and tracking system with only two pixels of backprojection error in the camera image.

The other part of the project focuses on soft tissue modeling for simulating virtual muscles. We have chosen mass-spring systems (MSS) to represent muscles due to their ease of implementation and real-time capability. However, the specification of system parameters for a MSS (masses, spring constants, mesh topology) is not straightforward. Our first focus was the determination of mesh topology based on genetic algorithms in the two dimensional case. The main idea for identification of MSS mesh topology was to compare the deformation of a training model with a known reference system. We were able to fully extract the topology of the reference model. Topologies characterizing linear isotropic as well as anisotropic deformations were successfully identified. The next step was to simultaneously estimate topology and spring constants. Moreover, we introduced FEM systems as reference and obtained MSM parameters describing the FEM deformations. We were able to determine appropriate elasticity values, so that the MSS deformations closely approximate the linear elastic FEM deformations.