Vision-Based Infotainment User Determination by Hand Recognition for Driver Assistance

We present a novel real-time computer-vision system that robustly discriminates which of the front-row seat occupants is accessing the infotainment controls. The knowledge of who is the user-that is, driver, passenger, or no one-can alleviate driver distraction and maximize the passenger infotainment experience. The system captures visible and near-infrared images of the front-row seat area in the vehicle. The algorithm uses a modified histogram-of-oriented-gradients feature descriptor to represent the image area over the infotainment controls and a support vector machine (SVM) and median filtering over time to classify each image to one of the three classes with 97.9% average correct classification rate. This rate was achieved over a wide range of illumination conditions, human subjects, and times of day. With an offset of 5 pixels in any direction, the rate could still be maintained at better than 85%. This approach represents an alternative to detecting and tracking the hand movements and then classifying the hands into the respective classes. This approach demonstrates the ability to achieve good classification rates, despite the presence of vast illumination changes of the vehicle environment.