Video synchronization from human motion using rank constraints

This paper presents a method of synchronizing video sequences that exploits the non-rigidity of sets of 3D point features (e.g., anatomical joint locations) within the scene. The theory is developed for homography, perspective and affine projection models within a unified rank constraint framework that is computationally cheap. An efficient method is then presented that recovers potential frame correspondences, estimates possible synchronization parameters via the Hough transform and refines these parameters using non-linear optimization methods in order to recover synchronization to sub-frame accuracy, even for sequences of unknown and different frame rates. The method is evaluated quantitatively using synthetic data and demonstrated qualitatively on several real sequences.