Object recognition and segmentation in videos by connecting heterogeneous visual features

We present an approach for model-free and instance-level object recognition and segmentation in cluttered scenes, based on heterogeneous visual features. The first contribution of this work addresses the description of the visual appearance of objects, by proposing the joint use of complementary features of different natures: on the one hand, a set of local descriptors based on interest points that have well-known interesting properties; on the other hand, a global descriptor based on a snake, providing a high-level description of the object shape. Our second contribution consists in efficiently structuring and connecting the visual features obtained, making possible the use of global descriptors without prior segmentation/detection. Our approach is compared to a classic one based on local descriptors only and is evaluated for video surveillance purposes over sequences involving 20 objects. We show that recognition is improved, and provides precise object segmentation, even with large occlusions. A real scenario of application to video surveillance of truck traffic validates the relevance of the approach.