DocumentCode
997975
Title
The uniform distribution as a universal prior
Author
Shulman, Nadav ; Feder, Meir
Author_Institution
Dept. of Electr. Eng.-Syst., Tel-Aviv Univ., Israel
Volume
50
Issue
6
fYear
2004
fDate
6/1/2004 12:00:00 AM
Firstpage
1356
Lastpage
1362
Abstract
In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.
Keywords
channel capacity; information theory; Z-channel; binary input channel; channel capacity; mutual information degradation; nonbinary input; uniform distribution; uniform prior; universal prior; Channel capacity; Degradation; Linear code; Maintenance; Monitoring; Mutual information; Transmitters; Upper bound; $Z$ -channel; Uniform distribution; universal prior;
fLanguage
English
Journal_Title
Information Theory, IEEE Transactions on
Publisher
ieee
ISSN
0018-9448
Type
jour
DOI
10.1109/TIT.2004.828152
Filename
1302318
Link To Document