FTV (free viewpoint television) creating ray-based image engineering

We have been developing ray-based 3D information systems that consist of ray acquisition, ray processing, and ray display. FTV (free viewpoint TV) based on the ray-space method is a typical example. FTV will bring an epoch-making change in the history of television because it enables us to view a distant 3D world freely by changing our viewpoints as if we were there. We constructed a real-time FTV including the complete chain from capturing to display. A new algorithm was developed to generate free viewpoint images. We need a new user interface for FTV to make full use of 3D information of FTV. Two types of user interface were made using 2D and automultiscopic displays with a head tracking system. We are creating "ray-based image engineering" through these works.