Title :
Interactive mobile visual search for social activities completion using query image contextual model
Author :
Ning Zhang ; Tao Mei ; Xian-Sheng Hua ; Ling Guan ; Li, Shipeng
Author_Institution :
Ryerson Multimedia Res. Lab., Ryerson Univ., Toronto, ON, Canada
Abstract :
Mobile devices are ubiquitous. People use their phones as a personal concierge not only discovering information but also searching for particular interest on-the-go and making decisions. This brings a new horizon for multimedia retrieval on mobile. While existing efforts have predominantly focused on understanding textual or a voice query, this paper presents a new perspective which understands visual queries captured by the built-in camera such that mobile-based social activities can be recommended for users to complete. In this work, a query image-based contextual model is proposed for visual search. A mobile user can take a photo and naturally indicate an object-of-interest within the photo via circle based gesture called “O” gesture. Both selected object-of-interest region as well as surrounding visual context in photo are used in achieving a search-based recognition by retrieving similar images based on a large-scale of visual vocabulary tree. Consequently, social activities such as visiting contextually relevant entities (i.e., local businesses) are recommended to the users based on their visual queries and GPS location. Along with the proposed method, an exemplary real application has been developed on Windows Phone 7 devices and evaluated with a wide variety of scenarios on million-scale image database. To test the performance of proposed mobile visual search model, extensive experimentation has been conducted and compared with state-of-the-art algorithms in content-based image retrieval (CBIR) domain.
Keywords :
content-based retrieval; image retrieval; interactive systems; mobile computing; multimedia systems; smart phones; trees (mathematics); visual databases; CBIR; GPS location; O gesture; Windows Phone 7 devices; built-in camera; content-based image retrieval; interactive mobile visual search; million-scale image database; mobile-based social activities; multimedia retrieval; object-of-interest; query image contextual model; search-based recognition; social activities completion; textual query; ubiquitous mobile devices; visual vocabulary tree; voice query; Context; Context modeling; Global Positioning System; Mobile communication; Search problems; Visualization; Vocabulary;
Conference_Titel :
Multimedia Signal Processing (MMSP), 2012 IEEE 14th International Workshop on
Conference_Location :
Banff, AB
Print_ISBN :
978-1-4673-4570-5
Electronic_ISBN :
978-1-4673-4571-2
DOI :
10.1109/MMSP.2012.6343447