Part Segmentation of Visual Hull for 3D Human Pose Estimation

In this paper we present an algorithm for estimating 3D pose of human targets using multiple, synchronized video streams obtained from a set of calibrated visual sensors. Our method uses 3D visual hull, reconstructed from multiview image silhouettes, to estimate skeleton and 3D pose of the human target. The key contribution of this work is to extend predictive human pose estimation algorithms used in the kinect gaming system to 3D visual hull data. In 3D space, viewpoint invariance is achieved by transforming the world reference frame to human centered reference frame. To do so, we first estimate the rigid body orientation and translation of the target from the shape of the visual hull. We then apply discriminative classifiers in the human centered reference frame to segment the 3D voxels of the visual hull into semantic part segments. The part clusters are then used to estimate a 3D pose that best aligns with the detected joint centers while conforming to the part non self-intersection constraints. Claims made in the work are supported by extensive experimental evaluation on both synthetic and real dataset.