Background extraction and long-term memory motion-compensated prediction for spatial-random-access-enabled video coding

Video streaming with virtual pan/tilt/zoom functionality allows the viewer to watch arbitrary regions of a high-spatial-resolution scene. A video coding scheme with random access to arbitrary regions of arbitrary zoom factors helps avoid transmission and/or decoding of the entire high-spatial-resolution video signal. The video coding scheme, proposed in our earlier work, creates a multi-resolution representation and uses P slices of H.264/AVC. The base layer, which provides a thumbnail overview of the entire scene, is encoded using motion-compensated prediction (MCP) among temporally successive frames. To provide efficient random access, we avoid MCP among temporally successive frames of the higher resolution layers. Instead, upward prediction from the reconstructed thumbnail frames is used for coding high-resolution P slices. In this paper, we show that background extraction and long-term memory motion-compensated prediction can reduce the bitrate by up to 85% while retaining efficient random access capability.