Learning mixtures of experts with a hybrid generative-discriminative algorithm

The hierarchical mixtures of experts (HME) model is a flexible model that stochastically partitions the feature space into sub-regions, in which simple surfaces can be fitted to data. However, there are issues with model selection during the learning of the HME model using Expectation-Maximization (EM) inference. In addition, the EM algorithm suffers from the well-known problem of local minima. In this paper, we present a hybrid generative-discriminative approach that inherits the flexibility of the HME while decomposing the learning process into a few simple steps. The proposed algorithm solves the model-selection problem and empirical experiments on public benchmark datasets show its advantages in classification accuracy and efficiency.