Depth SEEDS: Recovering incomplete depth data using superpixels

Depth sensors have become increasingly popular in interactive computer vision applications. Currently, most of these applications are limited to indoor use. Popular IR-based depth sensors cannot provide depth data when exposed to sunlight. In these cases, one can still obtain depth information using a stereo camera set up or a special outdoor Time-of-Flight camera, at the cost of a reduced quality of the depth image. The resulting depth images are often incomplete and suffer from low resolution, noise and missing information. The aim of this paper is to recover the missing depth information based on an extension of SEEDS superpixels [11]. The superpixel segmentation algorithm is extended to take depth information into account where available. The approach takes advantage of the boundary-updating property of SEEDS. The result is a clean segmentation that recovers the missing depth information in a low-quality depth image. We test the approach outdoors on an interactive urban robot. The system is used to segment a person in front of the robot, and to detect body parts for interaction with the robot using pointing gestures.