Data fusion in 3D vision using a RGB-D data via switching observation model and its application to people tracking

In this paper, we propose a new method for 3D people tracking with RGB-D observations. The proposed method fuses RGB and depth data via a switching observation model. Specifically, the proposed switching observation model intelligently exploits both final detection results and raw signal intensity in a complementary manner in order to cope with missing detections. In real-world applications, the detector response to RGB data is frequently missing. When this occurs the proposed algorithm exploits the raw depth signal intensity. The fusion of detection result and raw signal intensity is integrated with the tracking task in a principled manner via the Bayesian paradigm and labeled random finite set (RFS). Our case study shows that the proposed method can reliably track people in a recently published 3D indoor data set.