Visual sentiment analysis for brand monitoring enhancement

Brand monitoring and reputation management are vital tasks in all modern business intelligence frameworks. However, recent related technologies rely mostly on the textual aspect of online content, in order to extract the underlying sentiment with respect to particular brands. In this work, we demonstrate the sentiment analysis method in the context of a brand monitoring framework, breaking the text-only barrier in the field. Towards this end, a wide range of visual features is extracted, some of which focus on the underlying semiotics and aesthetics of the images. In addition, we employ textual information embedded in the images under study, by adopting text mining techniques that focus on extracting sentiment. We evaluate the classification task for the particular binary task (negative vs positive sentiment) and propose a fusion approach that combines the two different modalities. Finally, the evaluation procedure has been carried out in the context of two different use cases, namely: (a) a general image sentiment classifier for brand and advertising images and (b) a brand-specific classification procedure, according to which the brand of the input images is known a-priori. Results have proven that the visual-based sentiment classification of brand and advertising information can outperform the respective text-based classification. In addition, fusing the two modalities leads to significant performance boosting.