Video nature considerations for multi-frame selection algorithm in H.264

H.264 allows motion estimation performing on multiple reference frames. This new feature improves the prediction accuracy of inter-coding blocks significantly. However, the coding gain comes at the cost of a much higher computational complexity. The reference software JM adopts full search scheme, and the computational complexity of motion estimation increases linearly with the number of allowed reference frames. In fact, the reduction of prediction residues is highly dependent on the nature of sequences, not on the number of searched frames. In this paper, with consideration of video nature and the available information from previous searched reference frames, an adaptive multi-frame selection algorithm (AMFSA) is proposed to speed up the matching process for multiple reference frames in the H.264 video coding system. The proposed algorithm can effectively reduce 63.6% on average.