Density estimation by entropy maximization with kernels

The estimation of a probability density function is one of the most fundamental problems in statistics. The goal is achieving a desirable balance between flexibility while maintaining as simple a form as possible to allow for generalization, and efficient implementation. In this paper, we use the maximum entropy principle to achieve this goal and present a density estimator that is based on two types of approximation. We employ both global and local measuring functions, where Gaussian kernels are used as local measuring functions. The number of the Gaussian kernels is estimated by the minimum description length criterion, and the parameters are estimated by expectation maximization and a new probability difference measure. Experimental results show the flexibility and desirable performance of this new method.