Real-Time Foreground Segmentation for the Moving Camera Based on H.264 Video Coding Information

Foreground segmentation for video frames has played an important role in many video applications, such as video surveillance, video indexing, etc. Due to most videos are compressed, foreground segmentation can benefit from utilizing such coding information and save much processing time. In this paper, we propose a real-time foreground segmentation algorithm for the moving camera based on the H.264 video coding information. In the proposed algorithm, we first utilize the relative global motion model to calculate the approximate global motion vector and get the motion vector difference of each block. Then, according to the block partition modes, we assign different weightings and apply spatio-temporal refinement to these motion vector differences for further improving the accuracy of segmentation results. Finally, we segment out the foreground blocks by an adaptive threshold. With the aid of H.264 video coding information, the proposed segmentation algorithm is more practical than many other methods based on spatial domain information in computational complexity.