• DocumentCode
    3663104
  • Title

    Optimal entropy estimation on large alphabets via best polynomial approximation

  • Author

    Yihong Wu;Pengkun Yang

  • Author_Institution
    Department of Electrical and Computer Engineering and the Coordinated Science Lab, University of Illinois at Urbana-Champaign, USA
  • fYear
    2015
  • fDate
    6/1/2015 12:00:00 AM
  • Firstpage
    824
  • Lastpage
    828
  • Abstract
    Consider the problem of estimating the Shannon entropy of a distribution on k elements from n independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of (k/n log k) + log2k/n. This implies the recent result of Valiant-Valiant [1] that the minimal sample size for consistent entropy estimation scales according to Θ(k/log k). The apparatus of best polynomial approximation plays a key role in both the minimax lower bound and the construction of optimal estimators.
  • Keywords
    "Entropy","Estimation","Polynomials","Approximation methods","Nickel","Random variables"
  • Publisher
    ieee
  • Conference_Titel
    Information Theory (ISIT), 2015 IEEE International Symposium on
  • Electronic_ISBN
    2157-8117
  • Type

    conf

  • DOI
    10.1109/ISIT.2015.7282570
  • Filename
    7282570
    • Link To Document
    https://search.isc.ac/dl/search/defaultta.aspx?DTC=49&DC=3663104