Optimal transform coding in the presence of quantization noise

The optimal linear Karhunen–Lo`eve transform (KLT) attains the minimum reconstruction error for a fixed number of transform coefficients assuming that these coefficients do not contain noise. In any real coding system, however, the representation of the coefficients using finite number of bits requires the presence of quantizers. In this paper, we formulate the optimal linear transform using a data model that incorporates quantization noise. Our solution does not correspond to an orthogonal transform and in fact, it achieves smaller mean squared error (MSE) compared to the KLT, in the noisy case. Like the KLT, our solution depends on the statistics of the input signal, but it also depends on the bit-rate used for each coefficient. Especially for images, based on our optimality theory, we propose a simple modification of the discrete cosine transform (DCT). Our coding experiments show peak signal-to-noise ratio (SNR) performance improvement over JPEG of the order of 0.2 dB with overhead less than 0.01 b/pixel.