The Unknown Spatial Quality of Dense Point Clouds Derived From Stereo Images

Is it possible to use stereo images to generate point clouds and to compute rigorous uncertainty maps? Currently, neither modern commercial photogrammetric software nor state-of-the-art algorithms are able to provide a spatial distribution of uncertainty. In this letter, we explain why this is the case, despite a high demand from the user community. Many applications would indeed benefit from the availability of error bars on each point, as uncertainties on derived models and quantities could be accurately predicted. For instance, change detection could be performed rigorously since the statistical significance of observed changes could be computed. In this letter, we focus on dense stereo methods. We first explain that it is not possible to derive reliable predictive uncertainties mainly due to matching and modeling errors. Our research shows that both intrinsic and practical limitations of the algorithms lead to unpredictable artifacts. Then, we focus on the use of empirical errors, showing that, despite the redundancy brought by multiview stereo, there is a fundamental limitation due to the unknown density of independent measurements. We think that these problems will represent a big challenge for the future, as these limitations cannot be addressed by algorithmic design, computational power, or imaging sensor technology.