An experimental comparative study on several soft and hard-cut EM algorithms for mixture of experts

Mixture of expert (ME) (Jacobs, Jordan and Nowlan, 1991) and EM algorithms are very popular in supervised learning. Previously, an alternative ME model (Xu, Jordan and Hinton, 1995) and a number of hard-cut EM algorithms for both original and alternative ME (Xu, 1996) are proposed by one of the present authors. In this paper, we try to conduct a systematic experimental comparison on the two models through their implementation in soft and hard-cut EM algorithms. The comparison is based on the aspects of (1) the number of converged experiments with satisfactory results, (2) the classification correctness, (3) the training and testing error and, (4) time required. Experimental results obtained illustrate that the soft and hard-cut EM algorithms for the alternative ME have the highest percentage of convergence and classification correctness, much smaller training and testing error when compared with those algorithms for the original ME. Moreover, it requires much fewer number of iteration for the alternative ME to converged than that for the original ME