Structure-guided manifold learning for video-based motion estimation

We present a new structure-guided joint gait pose manifold (JGPM) that represents gait kinematics by two variables. One is the pose to denote a series of stages in a walking cycle and the other is the gait to reflect the individual walking styles. Coupling pose and gait variables in the same latent space, such as a torus-like JGPM, was shown promising and effective for video-based motion estimation. However, the two-step learning used in torus-like JGPM is computationally expensive and it separates the optimization of pose and gait variables. This work overcomes the limitations of the previous method by developing a new structure-guided JGPM that is able to jointly optimize four variables in the same latent space, leading to a much compact parameter set while sustaining a comparable performance on video-based motion estimation, as well as a great potential for large-scale learning.