Alternative Transform Based on the Correlation of the Residual Signal

An alternative transform based on the correlation of the residual signal is proposed for the improvement of the H.264/AVC coding efficiency. To make use of this alternative transform in video coding, discrete sine transform and integer sine transform are used alternately with integer cosine transform in order to greatly compact the energy of the signal when the correlation coefficients of the signal are relatively low. Therefore, the discrete sine transform and the integer sine transform are suggested to be used in conjunction with the integer cosine transform in H.264/AVC. The alternative transform that selects the optimal transform between two transforms by using rate-distortion optimization in H.264/AVC shows a coding gain compared with H.264/AVC. Based on the experimental results, the proposed method with the discrete sine transform achieves a BD-PSNR gain of up to 0.68 dB compared to JM 10.2 at relatively high bitrates.