Automatic Retrieval of Optimal Texture from Aerial Video for Photo-realistic 3D Visualization of Street Environment

This paper presents an approach of automatic retrieval of optimal texture from aerial video for photo-realistic 3D visualization of street environment. First, problems of using aerial video as texture source are summarized and based on that, a set of criterions for the determination of optimal texture are proposed; After that, texture ortho-rectification method based on 3D building model is developed to eliminate texture distortion due to inherent perspective projection; Next, the normalized cross-correlation (NCC) operation is applied to ortho-rectified texture sub-windows to select a pair of texture with least occlusion as two candidates of optimal texture; Finally, optimal texture is determined from two candidates based on the estimation of area of each texture view triangle (TVT), which is the right-angled triangle consisting of camera project position, geometry center of building wall and its normal vector. 170 buildings with 1015 walls, located in the 1.7 km long test area Ginzav Street, Tokyo, Japapen, are processed to obtain optimal texture from three strip of aerial video captured by helicopter. The experimental results demonstrated that the proposed approach increase 8 times more efficiency and effectiveness than manual operation for urban texture reconstruction.