An entertainment robot based on head pose estimation and facial expression recognition

In this paper, we propose a novel entertainment robot system that can act/react to humans. In the entertainment robots, how to act/react is the fundamental challenge. In general, it is very difficult for robots to automatically decide their behavior that human want to. Therefore entertainment robots with operator´s assistance are quite effective for designing practical system. Our proposed system is based on the computer vision technologies: head pose estimation and facial expression recognition. Our system leads to the commercially available entertainment robot to act/react to humans based on puppeteering by the operators head motion instead of autonomous behavior.