Extrinsic calibration of an RGB camera to a 3D imaging sonar

The introduction of low-cost RGB-depth (RGB-D) sensors have led to a diversity of algorithms for robust 3D scene reconstruction under controlled settings, but the underwater realization of such algorithms has been hampered by the constrained performance of most RGB-D sensors in water. We explore the possibility of fusing a point cloud generated from a high-frequency, mechanically scanned 3D imaging sonar with visual data from a camera to create a rich 3D representation of objects in the water column. A state-of-the-art algorithm for depth sensor-to-camera registration utilizing concurrent images of spherical targets is adapted, and the resulting alignment is used to combine sonar and visual imagery.