Sampling-based depth recovery in dynamic environments with motion assistance

This paper extends one state-of-the-art filter-based depth recovery method in dynamic environments with motion assistance and sampling. Our method fuses a depth data stream and a visual image stream, well calibrated, to achieve a high quality depth representation of the dynamic environments. For motion assistance, we use the optical flow between visual images to calculate the position of each point in target frame, and use the coordinates of this position as our new features named as Projection Coordinates (PC). After using these new features together with classical features (such as color, image coordinates, and time) to build a whole feature space, we propose a sampling-based High Dimensional Gaussian Filter (HDGF) framework for depth recovery. For any target point, we first find its k nearest neighbors in the feature space, which have corresponding known depth values, and calculate its distance to each neighbor. Then, we use a HDGF to integrate these k distances and k known depth values, to estimate the depth of the target point. Our method shows significant advantages in both qualitative and quantitative comparisons with the state-of-the-art method.