Title :
On the role of mismatch in rate distortion theory
Author_Institution :
Inf. Syst. Lab., Stanford Univ., CA, USA
Abstract :
Using a codebook C, a source sequence is described by the codeword that is closest to it according to the distortion measure d0(x,xˆ0). Based on this description, the source sequence is reconstructed to minimize the distortion measured by d1(x,xˆ1), where in general d1(x,xˆ1)≠d0(x,xˆ0 ). We study the minimum resulting d1(x,xˆ1 )-distortion between the reconstructed sequence and the source sequence as we optimize over the codebook subject to a rate constraint. Using a random coding argument we derive an upper bound on the resulting distortion. Applying this bound to blocks of source symbols we construct a sequence of bounds which are shown to converge to the least distortion achievable in this setup. This solves the rate distortion dual of an open problem related to the capacity of channels with a given decoding rule-the mismatch capacity. Addressing a different kind of mismatch, we also study the mean squared error description of non-Gaussian sources with Gaussian codebooks. It is shown that the use of a Gaussian codebook to compress any ergodic source results in an average distortion which depends on the source via its second moment only. The source with a given second moment that is most difficult to describe is the memoryless zero-mean Gaussian source, and it is best described using a Gaussian codebook. Once a Gaussian codebook is used, we show that all sources of a given second moment become equally hard to describe
Keywords :
Gaussian processes; approximation theory; channel capacity; convergence of numerical methods; memoryless systems; minimisation; random processes; rate distortion theory; sequences; source coding; Gaussian codebooks; average distortion; capacity; codebook; codeword; decoding rule; distortion measure; ergodic source; mean squared error description; memoryless zero-mean Gaussian source; mismatch; mismatch capacity; nonGaussian sources; random coding; rate distortion dual; rate distortion theory; reconstructed sequence; second moment; source sequence; source symbols; Books; Channel capacity; Decoding; Distortion measurement; Encoding; Information systems; Laboratories; Rate distortion theory; Rate-distortion; Upper bound;
Conference_Titel :
Information Theory, 1995. Proceedings., 1995 IEEE International Symposium on
Conference_Location :
Whistler, BC
Print_ISBN :
0-7803-2453-6
DOI :
10.1109/ISIT.1995.531867