A new hybrid fast mode decision method for H.264/AVC intra coding

A new hybrid fast mode decision method for H.264/AVC intra coding is proposed in this paper. In the proposed algorithm, edge based filters are applied to some samples distributed in the block and each directional mode is related to a filtering result. After that, the absolute sum of Hadamard Transformed coefficients of the filtering results is utilized to help selecting the candidate prediction modes. In addition, local information is employed by using Most Probable Mode (MPM) from neighboring coded blocks which can significantly mitigate the increase of coding bits. Experimental results demonstrate that the proposed method is able to provide a better coding performance than existing algorithms.