Real-time image-based rendering system with random access CMOS imagers

We investigate a real-time image-based rendering (IBR) system. IBR enables one to synthesize a 3D arbitrary view from a 2D array of input images. However, conventional systems have a difficulty to handle dynamic scenes because of the heaviness of the input. We design an IBR system with random access image sensors. In the system, each image sensor outputs only the pixels that are externally selected, so that the total readout data from the camera array is reduced to the necessary and sufficient condition for the rendering. We develop a prototype of random access image sensor by the CMOS process. A prototype of IBR system with 16 of this sensors was constructed and synthesized arbitrary-view at 60 frames/second.