Title :
The entropy theorem via coding bounds
Author :
Shields, Paul C.
Author_Institution :
Dept. of Math., Toledo Univ., OH, USA
fDate :
11/1/1991 12:00:00 AM
Abstract :
The entropy theorem (also known as the Shannon-McMillan-Breiman theory or the asymptotic equipartition theorem) asserts that, for a stationary ergodic finite alphabet process, the sequence-(1/n)log p(x1n) converges almost surely to the entropy-rate H of the process. The entropy theorem has been used to establish asymptotic bounds on the performance of noiseless codes. Here, the coding theorems are established without using the entropy theorem, and the coding theorems are then used to prove the entropy theorem. The principle feature is the direct use of coding ideas to obtain the entropy theorem
Keywords :
encoding; entropy; information theory; Shannon-McMillan-Breiman theory; asymptotic bounds; asymptotic equipartition theorem; coding bounds; entropy theorem; noiseless codes; stationary ergodic finite alphabet process; Binary sequences; Codes; Data compression; Entropy; H infinity control; Length measurement; Mathematics;
Journal_Title :
Information Theory, IEEE Transactions on