Rate-Distortion Optimized Progressive 3D Reconstruction from Multi-view Images

Most of the previous multi-view reconstruction algorithms focus on minimizing reconstruction distortion, i.e. reconstructing a 3D model as close to the real object as possible, and rely on subsequent simplification process to control the reconstruction rate for practical usage. In this paper, in addition to reconstruction distortion, we directly consider the reconstruction rate in wide-baseline multi-view reconstruction. In particular, we develop a novel rate-distortion efficient multi-view reconstruction system, which consists of two major components: the PDE-based mesh optimization and the local subdivision based mesh refinement. Although both components are prior arts, to achieve rate-distortion optimization, new ingredients are proposed including non-uniform sample distribution for efficient and accurate surface distortion approximation in mesh optimization and non-uniform vertex distribution in mesh refinement. From a set of calibrated and segmented wide baseline images, our developed system is able to reconstruct the optimal 3D mesh that achieves minimum reconstruction distortion at any rate within a certain range. Experimental results show that our proposed system significantly outperforms the original PDE-based reconstruction method in both rate-distortion performance and visual quality.