Acceleration of rate-distortion optimized quantization for H.264/AVC

Rate-distortion optimized quantization improves the coding performance of video compression. However, the search process involved in most existing methods is computationally expensive. In this paper, we present a novel method for accelerating the rate-distortion optimized quantization process for an H.264/AVC video encoder. The acceleration is achieved by using a rate model of entropy coding to directly solve the rate-distortion optimization problem. Compared with the H.264/AVC reference encoder, our method achieves an average 5.6% bitrate reduction for IBBP GOP coding structure, with less than 0.1% increase of total encoding time.