Prediction for human motion tracking failures

We propose a new and effective method of predicting tracking failures and apply it to the robust analysis of gait and human motion. We define a tracking failure as an event and describe its temporal characteristics using a hidden Markov model (HMM). We represent the human body using a three-dimensional, multicomponent structural model, where each component is designed to independently allow the extraction of certain gait variables. To enable a fault-tolerant tracking and feature extraction system, we introduce a single HMM for each element of the structural model, trained on previous examples of tracking failures. The algorithm derives vector observations for each Markov model using the time-varying noise covariance matrices of the structural model parameters. When transformed with a logarithmic function, the conditional output probability of each HMM is shown to have a causal relationship with imminent tracking failures. We demonstrate the effectiveness of the proposed approach on a variety of multiview video sequences of complex human motion.