Title :
Undersmoothed Kernel Entropy Estimators
Author :
Paninski, Liam ; Yajima, Masanao
Author_Institution :
Dept. of Stat., Columbia Univ., New York, NY
Abstract :
We develop a ldquoplug-inrdquo kernel estimator for the differential entropy that is consistent even if the kernel width tends to zero as quickly as 1/N, where N is the number of independent and identically distributed (i.i.d.) samples. Thus, accurate density estimates are not required for accurate kernel entropy estimates; in fact, it is a good idea when estimating entropy to sacrifice some accuracy in the quality of the corresponding density estimate.
Keywords :
differential equations; entropy; differential entropy; independent and identically distributed samples; undersmoothed kernel entropy estimators; Density measurement; Engineering profession; Entropy; Estimation error; Histograms; Kernel; Length measurement; Mutual information; Probability distribution; Smoothing methods; Approximation theory; bias; consistency; density estimation; distribution-free bounds;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2008.928251
