Title :
Universal Estimation of Erasure Entropy
Author :
Yu, Jiming ; Verdú, Sergio
Author_Institution :
Dept. of Electr. Eng., Princeton Univ., Princeton, NJ
Abstract :
Erasure entropy rate differs from Shannon´s entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, consistent universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Simulation results for those algorithms applied to Markov sources, tree sources, and English texts are compared to those obtained by fixed-order plug-in estimators with different orders.
Keywords :
Markov processes; data compression; entropy codes; tree codes; Markov source; Shannon entropy; context-tree weighting algorithm; data compression; erasure entropy estimation; universal algorithm; Algorithm design and analysis; Buildings; Context modeling; Data compression; Decoding; Entropy; Helium; Information rates; Information theory; Noise reduction; Bidirectional context tree; context-tree weighting; data compression; entropy rate; universal algorithms; universal modeling;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2008.2008117
