Toward low latency gesture control using smart camera network

Real-world gesture controlled applications are not yet widely present mainly due to strong practical constraints. As a step toward realizing a practical, affordable, low-power, real time, low-latency gesture control, we present a smart camera system and an algorithm for upper body pose reconstruction implemented on the system. A single-instruction multiple-data (SIMD) processor on a smart camera platform is used to detect person head and hands. The detected hand and head candidate positions are then transmitted to a central processor (a PC) where the data is combined and final decisions are made. Implementation of a computer vision algorithm on the SIMD camera processor is presented. We also describe the whole wireless smart camera system and analyze the performance and practical issues.