• DocumentCode
    997975
  • Title

    The uniform distribution as a universal prior

  • Author

    Shulman, Nadav ; Feder, Meir

  • Author_Institution
    Dept. of Electr. Eng.-Syst., Tel-Aviv Univ., Israel
  • Volume
    50
  • Issue
    6
  • fYear
    2004
  • fDate
    6/1/2004 12:00:00 AM
  • Firstpage
    1356
  • Lastpage
    1362
  • Abstract
    In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.
  • Keywords
    channel capacity; information theory; Z-channel; binary input channel; channel capacity; mutual information degradation; nonbinary input; uniform distribution; uniform prior; universal prior; Channel capacity; Degradation; Linear code; Maintenance; Monitoring; Mutual information; Transmitters; Upper bound; $Z$-channel; Uniform distribution; universal prior;
  • fLanguage
    English
  • Journal_Title
    Information Theory, IEEE Transactions on
  • Publisher
    ieee
  • ISSN
    0018-9448
  • Type

    jour

  • DOI
    10.1109/TIT.2004.828152
  • Filename
    1302318