Integrating physiological measurements with Avatars for effective identification technologies

Recent research in deception detection has focused on the use of Avatars, where an animated computer-generated talking head (an embodied agent) interviews a subject by asking a series of questions. The Avatar then records the ensuing verbal responses in an audio file for further analysis. The use of an Avatar can potentially mitigate issues related to biases (e.g., cultural, economical, personality related etc.) encountered when a human agent interviews a subject. In this work, we extend the capabilities of an existing Avatar system, by designing a method to capture the physiological state of the subject via Electrodermal Activity (EDA), photoplethysmography (PPG), and thermal imaging sensors during the course of the interview. The goal of this exercise is to synchronize the recorded audio responses to the Avatars generated questions with physiological measurements (e.g., facial thermal imaging recording) of the subject thereby facilitating a broader range of analysis leading to effective deception detection and human identification measurements. For example, thermal-based face images can be used to verify the identity of an interviewee.