Fast calculation of 8×8 integer DCT in the software implementation of H.264/AVC

In order to achieve higher compression performance the amendment called Fidelity Range Extension (FRExt) was added to the H.264 Advanced Video Coding (AVC) standard which uses both 4×4 and 8×8 integer discrete cosine transform (DCT) adaptively in the high profiles. This led to additional complexity of the initial version of the H.264/AVC encoder which had substantially high computational complexity. In this paper we propose a new algorithm which reduces the computational complexity for software implementation of horizontal 8×8 Integer DCT by more than 25 percent. Simulation results indicate 22 percent reduction in the computation time by using the proposed algorithm.