Building Telepresence Systems: Translating Science Fiction Ideas into Reality

Many people feel that at some time in the distant future it will be possible to see and interact with remote individuals so realistically that they will appear to be standing next to us, that it will also be possible for physicians to look inside their patients and see organs and tumors, as if they possessed Superman’s X-ray vision. Although we are far from the realization of such dreams, we are witnessing some encouraging progress toward them. For example, we understand that two aspects common to these systems are a) the acquisition and synthesis of complex 3D information (whether from cameras at a distant scene or medical imaging devices inside a patient), and b) reconstruction and presentation of the 3D information to the observer (whether a distant collaborator or a nearby physician). Early results from several institutions are encouraging. It is now possible to walk around distant scenes, although the visual data still needs to be preprocessed, the scene still needs to be static and the reconstruction still has gaps. It is possible to look inside patients, although with crude ultrasound imaging and with very limited visualization, or not naturally from the physician’s own point of view. It is not yet possible to view any of this kind of information with high degree of immersive 3D realism without wearing cumbersome visualization aids. Truly compelling realization of these long-held dreams will take careful analysis of the remaining problems, creative thinking about new approaches, and innovative and sustained development of the required acquisition and display technologies.