Bridging computer vision and social science: A multi-camera vision system for social interaction training analysis

We investigate the use of a vision-based system capable of estimating social states such as rapport and hostility. We study the correlation between interpretations automatically generated by our system and those reported by social scientists. Our multi-camera vision system collects visual cues including location (proximity), motion, pose, gaze, and facial expressions in real-time from multiple subjects moving freely in an unconstrained environment. We performed experiments on 80+ subjects. Preliminary regression analysis suggests high correlation between machine distilled time series signals and assessments made by human experts.