Joint gait-pose manifold for video-based human motion estimation

We propose a joint gait-pose manifold (JGPM) for human motion modeling that unifies two motion-related variables, i.e., pose (a specific posture in a walking cycle) and gait (an individual walking style), into one manifold representation. JGPM is able to capture the motion variability both across different poses and among multiple gaits simultaneously. We develop a new Gaussian processes (GP)-based dimension reduction algorithm to learn a torus-like JGPM that balances an ideal manifold structure and the intrinsic data structure. The learned JGPM outperforms existing GP-based methods in terms of the capability of gait interpolation. Also, JGPM is applied to video-based motion estimation in a particle filtering framework. Our algorithm is trained from the CMU Mocap data and tested on the Brown HumanEva dataset, and experimental results confirm the effectiveness of the proposed methods.