Multimodal Affective User Interface Using Wireless Devices for Emotion Identification

Emotion is a fundamental, essential, and critical function of human brain. It´s desirable to create intelligent computer systems to make emotion identification accurate and reliable. Firstly, we discuss the importance of emotion research. Then, we report on our efforts in developing multimodal affective user interface system used in emotion identification. There are different modalities of synchronous inputs from the user using wireless sensors and multimedia computer sensing, including physiological signals, facial expressions, speech signals, and behavioral video stream. The physiological signals are collected using wireless sensors with Bluetooth technology. The images of facial expression and the ongoing videos of behavior and gesture are recorded by several cameras connected to different controlling computers. All these signals and experiments are collected synchronously in real time to make multimodal signal analysis convenient. Finally, we discuss usages of this system in emotion assessment, learning and training