Real-time estimation of light source environment for photorealistic augmented reality

This paper proposes a vision-based augmented reality system with correct representation of attached and cast shadows. To realize a seamless augmented reality system, we need to resolve a number of problems. Especially, the geometric and photometric registration problems are important. These problems require to estimate the positions of light sources and user´s viewpoint. The proposed system resolves the problems using a 3D marker which combines a 2D square marker and a mirror ball. The 2D marker and the mirror ball are used to estimate the relationship between the real and virtual worlds and the positions of light sources in the real world, respectively.