DocumentCode :
850492
Title :
The entropy theorem via coding bounds
Author :
Shields, Paul C.
Author_Institution :
Dept. of Math., Toledo Univ., OH, USA
Volume :
37
Issue :
6
fYear :
1991
fDate :
11/1/1991 12:00:00 AM
Firstpage :
1645
Lastpage :
1647
Abstract :
The entropy theorem (also known as the Shannon-McMillan-Breiman theory or the asymptotic equipartition theorem) asserts that, for a stationary ergodic finite alphabet process, the sequence-(1/n)log p(x1n) converges almost surely to the entropy-rate H of the process. The entropy theorem has been used to establish asymptotic bounds on the performance of noiseless codes. Here, the coding theorems are established without using the entropy theorem, and the coding theorems are then used to prove the entropy theorem. The principle feature is the direct use of coding ideas to obtain the entropy theorem
Keywords :
encoding; entropy; information theory; Shannon-McMillan-Breiman theory; asymptotic bounds; asymptotic equipartition theorem; coding bounds; entropy theorem; noiseless codes; stationary ergodic finite alphabet process; Binary sequences; Codes; Data compression; Entropy; H infinity control; Length measurement; Mathematics;
fLanguage :
English
Journal_Title :
Information Theory, IEEE Transactions on
Publisher :
ieee
ISSN :
0018-9448
Type :
jour
DOI :
10.1109/18.104325
Filename :
104325
Link To Document :
بازگشت