Exploring 3-D Reconstruction Techniques: A Benchmarking Tool for Underwater Robotics

When the results of research in the field of robotics are presented to the scientific community, the same question is asked repeatedly: Are the results really reproducible? Regarding benchmarking issues, some technological areas where complex mechatronic devices, such as robots, have a central role are very far from other research areas like physics or chemistry, to name but a few, where reproducibility is always mandatory. Aside from mechatronic complexities, the comparison between two different algorithms in the same conditions is influenced by the experimental validation scenario. In underwater environments, the difficulties for benchmarking characterization increase substantially. This is especially true when the test bed is the sea where uncertainty is high. It is the aim of this article to present a software tool which enables a comparison between two different algorithms when the algorithms are being used to solve the same problem in water tank conditions. This is a preliminary stage before the final validation on the seabed. The evaluated algorithms fall into the three-dimensional (3-D) image reconstruction context, as a prior step to their autonomous manipulation. Performance results are presented for both simulation and real water tank conditions.