Title :
Redundancy of universal coding, Kolmogorov complexity, and Hausdorff dimension
Author :
Takahashi, Hayato
Author_Institution :
Inst. of Stat. Math., Tokyo, Japan
Abstract :
We study asymptotic code lengths of universal codes for parametric models. We show a universal code whose code length is asymptotically less than or equal to that of the minimum description length (MDL) code. Especially when some of the parameters of a source are not random reals, the coefficient of the logarithm in the formula of our universal code is less than that of the MDL code. We describe the redundancy in terms of Kolmogorov complexity and Hausdorff dimension. We show that our universal code is asymptotically optimal in the sense that the coefficient of the logarithm in the formula of the code length is minimal. Our universal code can be considered to be a natural extension of the Shannon code and the MDL code.
Keywords :
Bayes methods; encoding; probability; redundancy; Bayes methods; Hausdorff dimension; Kolmogorov complexity; MDL code; Shannon code; apriori probability; asymptotic code length; minimum description length; parametric model; stochastic complexity; universal code; Fractals; Information theory; Materials science and technology; Mathematics; Maximum likelihood estimation; Parametric statistics; Physics; Stochastic resonance; Upper bound; A priori probability; Bayes methods; Hausdorff dimension; Kolmogorov complexity; MDL; minimum description length; stochastic complexity; universal coding;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2004.836663