An efficient inter mode decision algorithm for H.264/AVC encoding optimization

In this paper, an efficient inter mode decision algorithm based on Compensated sum of Absolute Transform Difference (CSATD) is proposed for H.264/AVC video standard. The main idea is to determine the best inter mode based on CSATD cost function instead of the rate-distortion (R-D) cost. This approach can avoid most of the computationally intensive processes in the H.264/AVC mode decision, such as integer transform, quantization, variable length coding and pixel-reconstruction. Besides, it can solve the problem of SAD, SATD-based cost function for H.264/AVC mode decision in biasing to the smaller block size modes, as they are normally achieving higher prediction accuracy. It is demonstrated by experimental results that the proposed CSATD algorithm can reduce 62% to 72% of the H.264/AVC total encoding time with negligible degradation in R-D performance.