Super-resolution de-fencing: Simultaneous fence removal and high-resolution image recovery using videos

In real-world scenarios, images or videos taken at public places using inexpensive low-resolution cameras, such as smartphones are also often degraded by the presence of occlusions such as fences/barricades. Finer details in images captured using such low-end equipment are lost due to blurring and under-sampling. Compounding this problem is missing data due to the presence of an intervening occlusion between the scene and the camera such as a fence. To recover a fence-free high-resolution image, we use videos of the scene captured by panning a hand-held camera and model the effects of various degradations. Initially, we obtain the spatial locations of the fence/occlusions and the global shifts of the degraded background image. The underlying high-resolution fence-free image is modeled as a discontinuity-adaptive Markov random field and its maximum a-posteriori estimate is obtained using an optimization approach. The advantage of using this prior is that high-frequency information is preserved during the reconstruction of the super-resolved image. Specifically, we use the fast graduated non-convexity algorithm to minimize a non-convex energy function. Experiments with both synthetic and real-world data demonstrate the efficacy of the proposed algorithm.