VRAPS: Visual Rhythm-based Audio Playback System

VRAPS (Visual Rhythm-based Audio Playback System) is an interactive multimedia application that uses a novel visual rhythm detection technique to allow a user to control the playback speed of an audio signal by moving in front of a video camera. As traditionally defined in the context of music, a beat represents a distinctive musical event such as the hitting of a drum or the start of a new melodic note. Similarly, we define a visual beat to indicate distinctive visual events such as rhythmic gestures and dance motions. In this paper, we propose a method for automatically detecting visual beats and tempo from a video signal. As a demonstration of this technique, we present an interactive, real-time audio playback system that continuously adjusts the playback speed of an audio signal based on the visual rhythm detected from a real-time video capture device.