Title :
On the role of pattern matching in information theory
Author :
Wyner, A.D. ; Ziv, Jacob ; Wyner, Abraham J.
fDate :
10/1/1998 12:00:00 AM
Abstract :
In this paper, the role of pattern matching in information theory is motivated and discussed. We describe the relationship between a pattern´s recurrence time and its probability under the data-generating stochastic source. We show how this relationship has led to great advances in universal data compression. We then describe nonasymptotic uniform bounds on the performance of data-compression algorithms in cases where the size of the training data that is available to the encoder is not large enough so as to yield the asymptotic compression: the Shannon entropy. We then discuss applications of pattern matching and universal compression to universal prediction, classification, and entropy estimation
Keywords :
data compression; entropy; information theory; pattern matching; prediction theory; probability; source coding; Shannon entropy; classification; data-generating stochastic source; encoder; entropy estimation; information theory; nonasymptotic uniform bounds; pattern matching; probability; recurrence time; training data; universal data compression; universal prediction; Data compression; Entropy; Information theory; Jacobian matrices; Length measurement; Pattern matching; Source coding; Statistical distributions; Stochastic processes; Training data;
Journal_Title :
Information Theory, IEEE Transactions on
