Minimum mean square error approximation of unknown probability distribution functions

In many applications it is desirable to approximate an unknown probability distribution by a finite expansion in terms of known functions. In this paper, a double stochastic approximation algorithm is presented which recursively computes the optimal coefficients to minimize the mean square error in the approximation. The only required data are independent samples from the unknown probability distribution function. The procedure is derived in a straightforward manner and is computationally very simple in comparison to the results of others. A simulation was performed and the results are presented for both one sample run and for an average over several runs. The convergence rate appears to be comparable with that obtained by more complicated procedures that were previously proposed.