Efficient Synthesis of Arbitrary Viewpoint Images Using 3-D Geometric Model and Mesh-Based Specular Reflection Tracing

A novel image synthesis method of an object at arbitrary viewpoint by using the 3-D geometric model and several camera images is proposed. Although the image-based rendering synthesizes the relatively accurate diffuse reflection component from the camera images from different viewpoints, it can not synthesize the specular reflection component with sufficient accuracy as the movement of the specular reflection component as the changes in viewpoints are not taken into consideration. In order to overcome this problem, we propose a method for synthesizing the arbitrary viewpoint images with the use of the displacement vectors of the specular reflection components. The displacement vectors are estimated by a hexagonal matching algorithm based on the triangle mesh model which was modified for arbitrary shaped images. In the hexagonal matching, the images synthesized from 3-D geometric model of a real object and camera images are used as reference image. The experiments show that the proposed method reconstructs the specular reflection components more accurately and also improves the coding performance compared to the conventional methods.