Transform coding in AVS2

This paper describes the transform coding in the second generation of Audio-video Coding Standard (AVS2). In AVS2, 16-bit integer DCT transform scheme is adopted, with the transform block size varies from 4×4 to 32×32, and the smaller transform kernels are completely embedded in the larger ones. To keep strong decorrelation capability of the transform, a principle is proposed in this paper to design the transform kernel by jointly minimizing DCT distortion, orthogonality and normalization. Furthermore, complexity of the transform is analyzed based on the proposed hybrid butterfly architecture, in which the transform is decomposed into a butterfly structure and a low-cost matrix multiplication. Experiment result shows that 25.7% additions and 71.0% shift operations can be saved by using this architecture compared with the partial butterfly one.