A survey of ground-truth in emotion data annotation

In machine learning, the collection of the training data for the inference algorithms is a step of utmost importance because it constitutes the basis for building a classification model. Furthermore, in emotion research, the collection of a ground-truth emotion set is a very hard task due to the difficulties accompanying the assessment of the emotions elicited. While laboratory experiments for testing an emotion recognition system fail to evoke fully naturalistic responses, ambulatory assessment carries its own ambiguities when it comes to the nature of the emotion elicited and context awareness. Moreover, the incongruity that exists between the perceived and experienced emotion leads to doubtful measurements annotations. This paper aims to solve the problem of lack of ground-truth in emotion data and consequently to advance the accuracy of emotion recognition systems. For that, it surveys the most recent and significant state of the art on emotion assessment through context-aware systems and pinpoints their major characteristics. Then, it digs in deep into the data acquisition and annotation stage and reveals the major criteria for obtaining a ground-truth emotion set.