• DocumentCode
    1779594
  • Title

    Ensemble estimation of multivariate f-divergence

  • Author

    Moon, Kevin R. ; Hero, Alfred O.

  • Author_Institution
    Dept. of EECS, Univ. of Michigan, Ann Arbor, MI, USA
  • fYear
    2014
  • fDate
    June 29 2014-July 4 2014
  • Firstpage
    356
  • Lastpage
    360
  • Abstract
    f-divergence estimation is an important problem in the fields of information theory, machine learning, and statistics. While several divergence estimators exist, relatively few of their convergence rates are known. We derive the MSE convergence rate for a density plug-in estimator of f-divergence. Then by applying the theory of optimally weighted ensemble estimation, we derive a divergence estimator with a convergence rate of O (1/T) that is simple to implement and performs well in high dimensions. We validate our theoretical results with experiments.
  • Keywords
    learning (artificial intelligence); statistical distributions; MSE convergence rate; convergence rates; density plug-in estimator; information theory; machine learning; multivariate f-divergence estimation; optimally weighted ensemble estimation theory; statistics; Convergence; Convex functions; Entropy; Estimation; Information theory; Kernel; Taylor series;
  • fLanguage
    English
  • Publisher
    ieee
  • Conference_Titel
    Information Theory (ISIT), 2014 IEEE International Symposium on
  • Conference_Location
    Honolulu, HI
  • Type

    conf

  • DOI
    10.1109/ISIT.2014.6874854
  • Filename
    6874854