Optical flow based super-resolution: A probabilistic approach

This paper deals with the computation of a single super-resolution image from a set of low-resolution images, where the motion fields are not constrained to be parametric. In our approach, the inversion process, in which the super-resolved image is inferred from the input data, is interleaved with the computation of a set of dense optical flow fields. The case of arbitrary motion presents several significant challenges. First of all, the super-resolution setting dictates that the optic flow computations must be very precise. Furthermore, we have to consider the possibility that certain parts of the scene, which are visible in the super-resolved image, are occluded in some of the input images. Such occlusions must be identified and dealt with in the restoration process. We propose a Bayesian approach to tackle these problems. In this framework, the input images are regarded as sub-sampled and noisy versions of the unknown high-quality image. Also, the input data is considered incomplete, in the sense that we do not know which pixels from the evolving super-resolution image are occluded in particular images from the input set. This will be modelled by introducing so-called visibility maps, which are treated as hidden variables. We describe an Expectation-Maximisation (EM) algorithm, which iterates between estimating values for the hidden quantities, and optimising the flow-fields and the super-resolution image. The approach is illustrated with a synthetic and two challenging real-world examples.