A pointing system based on acoustic position estimation and gravity sensing

This paper presents a novel pointing system for distant displays based on an acoustic position estimation technique in conjunction with a gravity sensor on a smartphone. The system is designed to interact with a distant large display such as a television set at home or digital signage in public. The system consists of a display, two loudspeakers set around the display, and a smartphone as a pointing device containing a microphone and a gravity sensor inside. The position of the pointer is theoretically determined by the position and direction of the smartphone, given by the three-dimensional acoustic position estimation. This smartphone-based system approximates the position and direction by the two-dimensional position of the microphone horizontally and the pitch angle from the gravity sensor vertically. The loudspeakers of the prototype system radiate burst signals of 20 kHz modulated by pre-windowed M-sequence, which are reproducible by normal television sets. The position of the smartphone is estimated at a frame rate of 23 Hz, and the pointer is rendered at a frame rate of over 100 Hz with linear interpolation and with a latency of 0.4 second.