Title :
Rate-distortion via Markov chain Monte Carlo
Author :
Jalali, Shirin ; Weissman, Tsachy
Author_Institution :
Dept. of Electr. Eng., Stanford Univ., Stanford, CA
Abstract :
We propose a new approach to lossy source coding. The idea is to sample a reconstruction sequence from a Boltzmann distribution associated with an energy function that incorporates the distortion between the source and reconstruction, the compressibility of the reconstruction, and the point sought on the rate-distortion curve. To sample from this distribution, we use a heat bath algorithm: Starting from an initial candidate reconstruction (say the original source sequence), at every iteration, an index i is chosen and the ith sequence component is replaced by drawing from the conditional probability distribution for that component given all the rest. At the end of this process, the encoder losslessly conveys the reconstruction to the decoder using universal lossless compression. An appropriate choice of the energy function leads to an algorithm whose complexity, in each iteration, is independent of the sequence length and only linearly dependent on a certain context parameter (which grows sub-logarithmically with the sequence length). The algorithm is universal: for any stationary ergodic source, it achieves the optimal rate-distortion performance in the limits of large number of iterations and sequence length. Initial experimentation shows promising performance in practice.
Keywords :
Boltzmann equation; Markov processes; Monte Carlo methods; decoding; iterative methods; probability; source coding; Boltzmann distribution; Markov chain Monte Carlo; conditional probability distribution; decoder; energy function; heat bath algorithm; iteration method; lossy source coding; rate-distortion; rate-distortion curve; reconstruction compressibility; reconstruction sequence; stationary ergodic source; universal lossless compression; Arithmetic; Boltzmann distribution; Distortion measurement; Entropy; Iterative decoding; Monte Carlo methods; Probability distribution; Rate-distortion; Simulated annealing; Source coding;
Conference_Titel :
Information Theory, 2008. ISIT 2008. IEEE International Symposium on
Conference_Location :
Toronto, ON
Print_ISBN :
978-1-4244-2256-2
Electronic_ISBN :
978-1-4244-2257-9
DOI :
10.1109/ISIT.2008.4595107