Title :
Limit results on pattern entropy
Author :
Orlitsky, Alon ; Santhanam, Narayana P. ; Viswanathan, Krishnamurthy ; Zhang, Junan
Author_Institution :
Dept. of Electr. & Comput. Eng., California Univ., San Diego, La Jolla, CA, USA
fDate :
7/1/2006 12:00:00 AM
Abstract :
We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones.
Keywords :
entropy codes; probability; random processes; asymptotic equipartition property; finite-entropy stationary processes; persymbol pattern entropy; probability; random process; Entropy; H infinity control; Image coding; Random processes; Source coding; Speech; Sufficient conditions; Upper bound; Asymptotic equipartition property; entropy rate; large alphabets; patterns; universal compression;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2006.876351
