Title :
The empirical distribution of good codes
Author :
Shamai, Shlomo ; Verdú, Sergio
Author_Institution :
Dept. of Electr. Eng., Technion-Israel Inst. of Technol., Haifa, Israel
Abstract :
Finding the input distribution that maximizes mutual information leads, not only to the capacity of the channel, but to engineering insights that tell the designer what good codes should be like. This is due to the folk theorem: The empirical distribution of any good code (i.e., approaching capacity with vanishing probability of error) maximizes mutual information. This paper formalizes and proves this statement
Keywords :
channel capacity; codes; probability; statistical analysis; channel capacity; empirical distribution; engineering; good codes; input distribution; mutual information; vanishing error probability; Channel capacity; Convergence; Decoding; Design engineering; Distributed computing; Entropy; Information theory; Memoryless systems; Mutual information; Statistical distributions;
Conference_Titel :
Information Theory, 1995. Proceedings., 1995 IEEE International Symposium on
Conference_Location :
Whistler, BC
Print_ISBN :
0-7803-2453-6
DOI :
10.1109/ISIT.1995.531115
