Markerless monocular motion capture using image features and physical constraints

We present a technique to extract motion parameters of a human figure from a single video stream. Our goal is to prototype motion synthesis rapidly for game design and animation applications. For example, our approach is especially useful in situations where motion capture systems are restricted in their usefulness given the various required instrumentation. Similarly, our approach can be used to synthesize motion from archival footage. By extracting the silhouette of the foreground figure and using a model-based approach, the problem is re-formulated as a local, optimized search of the pose space. The pose space consists of 6 rigid body transformation parameters plus the internal joint angles of the figure. The silhouette of the figure from the captured video is compared against the silhouette of a synthetic figure using a pixel-by-pixel, distance-based cost function to evaluate goodness-of-fit. For a single video stream, this is not without problems. Occlusion and ambiguities arising from the use of a single view often cause spurious reconstruction of the captured motion. By using temporal coherence, physical constraints, and knowledge of the anatomy, a viable pose sequence can be reconstructed for many live-action sequences.