Web-based vs. controled environment: About the reliability of stimuli ratings in human-robot interaction

In several research areas, e.g. in the field of human-robot interaction, ratings or questionnaires are applied using offline and online methods. An argument for the use of online methods is the efficiency. By using the Internet, data can be collected much faster than in an offline experiment and the administration effort is very low. The goal of our study was to find out, if there is a difference in accuracy between an online and an offline rating task of human and robot emotional facial expressions. Results indicate, that emotional expressions are best recognized in humans (versus robots) and in the offline (versus online) condition. Furthermore, the influence of the emotional category on the accuracy rate varies between conditions. Therefore, we discuss environmental factors of online experiments that are difficult to control as main reasons for these results. We conclude that online rating studies should always be combined with more reliable offline evaluations.