Modified self-organizing mixture network for probability density estimation and classification

In this paper, a modified algorithm based on the Self-organizing Mixture Network (SOMN) is proposed to learn arbitrarily complex density functions accurately and effectively. The algorithm is derived based on the criterion of minimizing the Kullback-Leibler divergence, maximum likelihood approach and self-organizing principle. It has the advantages of stochastic approximation method such as fewer local optima and faster convergence speed and the prominent properties of the neural networks such as good generalization ability, and overcomes the limitations of the SOMN. These greatly improve its stability, applicability and computation performance. This algorithm also simplifies the competitive and cooperative mechanism used in the self-organizing map (SOM). This lets it has a well-defined objective function and helps to provide a general proof of convergence. Experiments show that this modified algorithm outperforms the Expectation-Maximization (EM) algorithm, the SOMN and the joint entropy maximization algorithm in estimation accuracy. It is far superior to the EM algorithm in terms of learning speed and computational cost. Experimental results show that when used to estimate large datasets, this algorithm is 30-80 times faster than the EM algorithm at least. Owing to its outstanding density estimation performance, this algorithm is very helpful to the construction of optimal classifiers. The effectiveness of the algorithm is demonstrated in several real-world applications.