Detail warping based video super-resolution using image guides

Video scaling is a crucial video processing component that finds a variety of applications in video display, enhancement and analysis. Thus, video scaling has been an important research topic for several years now. With the emergence of hybrid-cameras that simultaneously capture a low-resolution (LR) video and periodic high-resolution (HR) stills, methods that use information in HR stills for enhancing the LR video are being studied. We present a multi-layer super-resolution (SR) method to rescale the input LR video with the help of a few HR still image guides. Resolution enhancement is achieved by blending the high frequency details into the input video frames that are scaled-up by a conventional, spatial scaling method. The missing high frequency details for each video frame are derived by spatio-temporal warping of the high frequency details extracted from HR still image guides. The Non-Local-Means (NLM) concept is used for computing the warping weights. The proposed SR method handles both global and local motions in a video, without requiring any explicit image registration. Experimental results comparing the proposed method with some of the existing video scaling techniques clearly points its strong potential.