Integrated target tracking and recognition via joint appearance-motion generative models

We address the problem of joint target tracking and recognition by incorporating both appearance and motion information via two generative models. Specifically, a non-linear tensor decomposition method is used to develop an appearance generative model for multi-pose target representation. In addition, a target-dependent kinematic model is invoked to capture different target dynamics. Both generative models are integrated in a graphical model to work together for joint estimation of the kinematics, pose, and identity of the target. A particle filter is developed for inference in the graphical model where a Kalman filter is embedded to improve the proposal generation by taking advantage of motion cues. Tests on simulated infrared sequences demonstrate the advantages and potential of the proposed approach for joint tracking and recognition.