A fast luminance intra 4×4 prediction mode decision method by statistical analysis of residual data in H.264/AVC

In H.264/AVC, intra prediction mode decision using rate distortion optimization (RDO) improves coding efficiency but requires high computational complexity. There is a close correlation between the best mode by RDO and the energy of the residual data. In this paper we propose a fast intra 4×4 block prediction mode decision method by statistical analysis of the relationship between RDO and residual data. The proposed method reduces the intra 4×4 block encoding time by about 57.4%, while decreasing coding gain by about 0.29%.