Remote Virtual Peripheral Framework: Enabling dynamically composed devices

As the number of mobile devices grow at a rapid pace, the heterogeneity in their sensing and user interaction capabilities is also increasing. Mobile devices are increasingly being equipped with sensors such as accelerometers, gyros and other gesture sensitive capabilities which propel user experiences beyond the usual keyboard-mouse-touch paradigm. A natural question is how can specific sensors of different devices inter-operate on-demand in a seamless fashion so that they can be composed together into a user interface providing a rich and natural user experience. For example, it should be possible to use the accelerometer in a mobile phone as an input device to control a video game running on a PC, or a high-definition camera attached to a television to act as the input video source for a video chat session running on a mobile phone. In this paper we present a system framework and architecture which enables the seamless interoperability and composition of sensing and user interaction capabilities of different devices into a unified user interface, in an application-agnostic manner.