Enabling User Interactions with Video Contents

Many people spend countless hours watching videos online or on TV. Current smart TV systems provide some basic two-way communications where users can interact with some features (e.g. browsing web, accessing social media, etc) provided by service providers. We would like to move beyond such primitive interactions and explore the possibility of allowing users to interact with video contents. For example, users can select objects shown in videos and place further queries on them. We start with exploring different state-of-the-art object detection and tracking techniques to obtain an object´s location in the video. Using the best performing tracking technique, we extract an object´s location in each frame and allow users to interact with the object using Microsoft Kinect. Finally, we have developed and compared a set of selection techniques that assist users to select moving objects in video. We conclude with guidelines for designing such interaction systems.