Supervised topic regression via experts

This paper focuses on the research issue of supervised topic models. Traditionally, an unsupervised topic model is typically supervised by incorporating a supervised linear operator. Although this kind of methods has successfully achieved supervised topic representations, as well as tractable computational complexity, the main limitation lies in that it assumes the topic data is linearly distributed. Therefore, when the practical data does not follow the linear property, the model cannot perform well. To solve this problem, a non-linear supervised topic model is proposed in this paper. Specifically, the mixture of experts (ME), as a kind of "divide and conquer", is utilized to deal with the regression of non-linear of topic data. We integrate the mixture of experts and unsupervised latent Dirichlet allocation (LDA) in a Bayesian manner. The proposed model can also reduce overfitting problem of ME with inputting high-dimensional data. An elegant learning algorithm is derived based on variational expectation maximization algorithm. Experimental results show that the proposed model has better predictive performances compared with usupervised topic models, and some state-of-the-art supervised topic regression models (including sLDA model and MedLDA model), on two textual datasets and one image dataset.