High-speed sensing system for depth estimation based on depth-from-focus by using smart imager

The paper proposes a new sensing system for depth estimation based on depth-from-focus (DFF). The DFF technique uses a search for the most in-focus position over a sequence of images taken by varying a camera parameter, "focus", little by little; therefore, high spatial and temporal resolution cannot be achieved because of the output bottleneck of the image sensor. In the proposed system, the limitation is improved by using a smart CMOS image sensor integrated with an in-focus detection function. We have designed and fabricated the smart sensor that operates at 1500 fps and the depth estimation can be done at 30 Hz and 50 depth steps. Moreover, an interpolation processing for correcting estimation errors is performed on an FPGA outside the sensor. We show some simulation results to demonstrate the effectiveness of the proposed system.