FPGA based system for real-time structure from motion computation

In this paper a system for real-time structure from motion computation is presented. The module is dedicated for a video based Advanced Driver Assistance System, where information about the surrounding environment and scene depth is often required to facilitate advanced applications. A heterogeneous System on a Chip solution, based on a FPGA device and an ARM processor is presented. The described hardware-software co-design is able to perform all steps required for structure from motion computation: feature extraction for every video frame, feature matching between two consecutive frames, finding the fundamental matrix, recovering the projection matrix and points triangulation. The design space exploration for the hardware part of the system is presented, which clearly points out the tradeoffs between resource usage, maximum clock and results accuracy. The system can process in real-time the images of 720 × 576 pixel resolution @ 50 fps. While the triangulation module allows for even high definition images processing and is able to compute results for up to 388 million points per second.