Mixture density estimation via Hilbert space embedding of measures

In this paper, we consider the problem of estimating a density using a finite combination of densities from a given class, C. Unlike previous works, where Kullback-Leibler (KL) divergence is used as a notion of distance, in this paper, we consider a distance measure based on the embedding of densities into a reproducing kernel Hilbert space (RKHS). We analyze the estimation and approximation errors for an M-estimator and show the estimation error rate to be better than that obtained with KL divergence while achieving the same approximation error rate. Another advantage of the Hilbert space embedding approach is that these results are achieved without making any assumptions on C, in contrast to the KL divergence approach, where the densities in C are assumed to be bounded (and away from zero) with C having a finite Dudley entropy integral.