Trilateral filter construction for depth map upsampling

In recent years, fusion camera systems that consist of color cameras and Time-of-Flight (TOF) depth sensors have been popularly used due to its depth sensing capability at real-time frame rates. However, captured depth maps are limited in low resolution compared to the corresponding color images due to physical limitation of the TOF depth sensor. Although many algorithms have been proposed, they still yield erroneous results, especially when boundaries of the depth map and the color image are not aligned. We therefore propose a novel kernel regression framework to generate the high quality depth map. Our proposed filter is based on the vector pointing homogeneous pixels that represents the unit vector toward similar neighbors in the local region. The vectors are used to detect misaligned regions between color edges and depth edges. Experimental comparisons with other data fusion techniques prove the superiority of the proposed algorithm.