Multi-view deep learning for image-based pose recovery

Image-based human pose recovery is usually conducted by retrieving relevant poses with image features. However, semantic gap exists for current feature extractors, which limits recovery performance. In this paper, we propose a novel method to recover 3D human poses from silhouettes. It is based on multiple feature fusion and deep learning. First, to fuse different types of features, we introduce manifold alignment with hypergraph Laplacian. Hypergraph Laplacian matrix is constructed with patch alignment framework. Second, multi-view description is applied to deep neural networks. In this way, the non-linear mapping from 2D images to 3D poses is learned and pose recovery can be achieved. Experimental results on the widely-used Human3.6m dataset show that the recovery error has been reduced by 10% to 20%, which demonstrates the effectiveness of the proposed method.