Abstract :
In this paper, we propose a revisitation of the topic of unique decodability and of some fundamental theorems of lossless coding. It is widely believed that, for any discrete source X, every ldquouniquely decodablerdquo block code satisfies E[l(X 1, X 2,..., X n)]gesH(X 1, X 2,..., X n) where X 1, X 2,..., X n are the first n symbols of the source, E[l(X 1, X 2,..., X n)] is the expected length of the code for those symbols, and H(X 1, X 2,..., X n) is their joint entropy. We show that, for certain sources with memory, the above inequality only holds when a limiting definition of ldquouniquely decodable coderdquo is considered. In particular, the above inequality is usually assumed to hold for any ldquopractical coderdquo due to a debatable application of McMillan´s theorem to sources with memory. We thus propose a clarification of the topic, also providing an extended version of McMillan´s theorem to be used for Markovian sources.
Keywords :
Markov processes; block codes; computational complexity; decoding; Markovian sources; McMillan theorem; block code; entropy; lossless coding; unique decodability; Associate members; Automation; Block codes; Decoding; Electronic mail; Entropy; Helium; Random variables; Source coding; Constrained sources; McMillan´s theorem; lossless source coding; minimum expected code length;
