Pedestrian localization by appearance matching and multi-mode filtering

This paper addresses the frame-to-frame data association and state estimation problems in localization of a pedestrian relative to a moving vehicle from a far infra-red video sequence. In a novel application of the hierarchical model-based motion estimation framework, we are able to solve the frame-to-frame data association problem as well as estimate a sub-pixel accurate height ratio for a pedestrian in two frames. To estimate the position and velocity of a pedestrian, instead of using a constant pedestrian height model, we propose a novel approach of using the interacting multiple-hypothesis-mode/height filtering algorithm. We present a method to calculate the probability of each mode from the estimated and measured pedestrian height ratios in images. These mode probabilities are then used to accurately estimate the pedestrian location by combining the mode based estimations. We demonstrate the effectiveness of our approach comparing it to a constant height model based approach on several IR sequences.