Robust method for real-time, continuous, 3D detection of obstructed faces in indoors environments

We address the need for robust detection of obstructed human features in complex environments, with a focus on intelligent surgical UIs. In our setup, real-time detection is used to find features without the help of local (spatial or temporal) information. Such a detector is used to validate, correct or reject the output of the visual feature tracking, which is locally more robust, but drifts over time. In operating rooms (OR), surgeon faces are typically obstructed by sterile clothing and tools, making statistical and/or feature-based face detection approaches ineffective. We propose a new method for face detection that relies on geometric information from disparity maps, locally refined by color processing. We have applied our method to a surgical mock-up scene, as well as to images gathered during real surgery. Running in a real-time, continuous detection loop, our detector successfully found 99% of target heads (0.1% false positive) in our simulated setup, and 98% of target heads (0.5% false positive) in the surgical theater