Multimodal Query Suggestion and Searching for Video Search

In this paper, we propose a multimodal query suggestion method for video search engine which can leverage multimodal processing to improve the quality of search results. When users type general or ambiguous textual queries, our system provides keyword suggestions and representative image examples in an easy-to-use dropdown manner which can help users specify their search intent more precisely and effortlessly. It is a powerful complement to initial queries. After the queries are formulated as multimodal query (i.e., text, image), the new queries are input to individual search models, such as text-based, concept-based and visual example-based search model. Then we apply multimodal fusion method to aggregate the above-mentioned several search results. The effectiveness of our system is evaluated by evaluations over a Web video data set.