Sentiment analysis of Chinese micro-blog based on multi-modal correlation model

Text, emoticons and images, various modalities have been used to express users´ feelings on social media, which significantly challenges traditional text-based sentiment analysis approaches. In this paper, we propose a Multi-modal Correlation Model (MCM) for multi-modal sentiment analysis. Compared with other multi-modal methods, MCM models hierarchical correlations among modalities, as well as between modalities and sentiments. Specifically, a probabilistic graphical model (PGM) is subsequently built upon the proposed MCM model, which considers the hierarchical correlations and preserves the classification ability of each modality. In order to compute the posterior probabilities of sentiments in PGM, we optimize the model by Maximum Likelihood Estimation. Experimental results demonstrate: 1) the hierarchical correlations among different modalities and sentiment; 2) the importance of hierarchical correlations to sentiment analysis.