Towards 3D human posture estimation using multiple kinects despite self-contacts

We present a marker-less human motion capture system that uses multiple RGB-D cameras to estimate the 3D posture of multiple people online at interactive rates in an indoor workspace measuring approximately 5 m × 5 m × 2 m. An interesting aspect of this work is how we handle the self-contact problem. We propose a novel multi-view voting scheme (MVS) to fuse measurements from different 2D or 3D algorithms. As a proof of concept, we present an MVS implementation that fuses optical flow images from each view and labels points in the current instance using the previously estimated posture. These labels allow us to trim invalid edges in a geodesic distance graph model and improve localization of geodesic extrema corresponding to the head, hands and feet for posture estimation. The system performs at ~8.3 Hz with a cumulative latency of ~570.40 ms and a projected median localization error of ~0.149 m. In addition, we propose a new multi-view Kinect and Vicon publicly accessible motion capture dataset for validation and benchmarks.