Efficient object retrieval from videos

We describe an approach to video object retrieval which enables all shots containing the object to be returned in a manner, and with a speed, similar to a Google search for text. The object is specified by a user outlining it in an image, and the object is then delineated in the retrieved shots. The method is based on three components: (i) an image representation of the object by a set of viewpoint invariant region descriptors so that recognition can proceed successfully despite changes in viewpoint, illumination and partial occlusion; (ii) the use of contiguous frames within a shot in order to improve the estimation of the descriptors and motion group object visual aspects; (iii) vector quantization of the descriptors so that the technology of text retrieval, such as inverted file systems, can be employed at run time. The method is illustrated on a full length feature film.