N-level quantization in HEVC

All transformed coefficients in a TU (Transform Unit) are equally quantized in the HEVC (High Efficiency Video Coding) standard. Such an equal quantization is not efficient because the transformed coefficients in the TU are not equally distributed. Moreover, the quantized coefficients scanning order method in the HEVC standard takes into account the distribution of the quantized coefficients in the TU and the entropy coding method is applied to the different scanning order. In this paper, transform coefficients in a TU are quantized as N levels considering an efficiency of the CABAC entropy encoding. N-level quantization and inverse quantization in each TU are carried out according to the scanning order. The proposed method is implemented on the HM6.0 without the RDOQ (Rate Distortion Optimized Quantization) encoder option. The proposed method shows about 0.3-0.6% Luma BD-rate gain.