Kernel-based mixture of experts models for linear regression

This paper proposes a novel kernel-based mixture of experts model for linear regression. The method is novel in that it formulates the mixture of experts model for linear regression so that kernel functions can be used. This allows the method to work directly in terms of kernels and avoids the explicit introduction of the feature vector, allowing one to use feature spaces of high, even infinite dimensionality. Other advantages of the model include the ability to take advantage of all the work related to kernels, a closed-form solution for maximization, as well as maintaining all the advantages of a linear expert. In this paper the supervised version is formulated. The model is verified and tested with simulated data. It was also found that the model had overall better performance than standard mixture of experts for regression on the well-known Boston Housing data set. Kernels used included polynomial, radial basis function and the ANOVA kernel.