Cabac Based Bit Estimation for Fast H.264 RD Optimization Decision

Rate distortion (RD) optimization, which is used to decide the best coding mode of a macroblock (MB) in H.264/AVC video coding standard, is a very efficient technique to improve coding performance. On the other hand, this RD optimization process also incurs a considerable complexity increase of the encoder. To reduce computational cost of RD optimization, fast bit estimation method can be adopted to avoid entropy coding computation during the mode decision process of H.264/AVC encoding [1] [2]. However, previous works [1][2] are all for CAVLC (Context Based Adaptive Variable Length Coding) based RD optimization process and are not accurate for CABAC (Context Based Adaptive Binary Arithmetic Coding) based RD optimization process. As known, compared with CAVLC, CABAC entropy coding provides higher coding efficiency at average bit rate saving of 9%-14% [3]. Hence, it is desirable to find a fast bit estimation method that is suitable for RD optimization with CABAC entropy coding. In this paper, we present such kind of fast bit estimation method. The experiments show that the proposed method can reduce around 30% of the RD optimization time averagely while maintaining the similar coding performance comparing to that of original H.264/AVC with RD optimization enabled.