Simultaneous color consistency and depth map estimation for radiometrically varying stereo images

In this paper, we propose a new method that infers accurate depth maps and color-consistent images between radiometrically varying stereo images, simultaneously. In general, stereo matching and performing color consistency between stereo images are a chicken-and-egg problem. Color consistency enhances the performance of stereo matching, while accurate correspondences from stereo disparities improve color consistency between stereo images. We devise a new iterative framework in which these two processes can boost each other. For robust stereo matching, we utilize the mutual information-based method combined with the SIFT descriptor from which we can estimate the joint pdf in log-chromaticity color space. From this joint pdf, we can estimate a linear relationship between the corresponding pixels in stereo images. Using this linear relationship and the estimated depth maps, we devise a stereo color histogram equalization method to make color-consistent stereo images which conversely boost the disparity map estimation. Experimental results show that our method produces both accurate depth maps and color-consistent stereo images even for stereo images with severe radiometric differences.