Mapping of endoscopic images to object surfaces via ray-traced texture mapping for image guidance in neurosurgery

A major limitation of the use of endoscopes in minimally invasive surgery is the lack of relative context between the endoscope and its surroundings. The purpose of this work is to map endoscopic images to surfaces obtained from 3D preoperative MR or CT data, for assistance in surgical planning and guidance. To test our methods, we acquired pre-operative CT images of a standard brain phantom from which object surfaces were extracted. Endoscopic images were acquired using a neuro-endoscope tracked with an optical tracking system, and the optical properties of the endoscope were characterized using a simple calibration procedure. Registration of the phantom (physical space) and CT images (pre-operative image space) was accomplished using markers that could be identified both on the physical object and in the pre-operative images. The endoscopic images were rectified for radial lens distortion, and then mapped onto the extracted surfaces via a ray-traced texture-mapping algorithm, which explicitly accounts for surface obliquity. The optical tracker has an accuracy of about 0.3 mm, which allows the endoscope tip to be localized to within (0.8+/-0.3) mm. The mapping operation allows the endoscopic images to be effectively "painted" onto the surfaces as they are acquired. Panoramic and stereoscopic visualization and navigation of the painted surfaces may then be reformed from arbitrary orientations, that were not necessarily those from which the original endoscopic views were acquired.