Comparative evaluation of robotic software integration systems: A case study

One might expect that after nearly 50 years of research in robot manipulation and mobile robotics the architectural design of robot systems has converged to a small set of best practice solutions, very much like in the area of operating systems, where the world is dominated by less then a handful systems. Quite the opposite is the case. It is only a small overstatement to say that almost every lab has brewed its own solution for robot control architecture, middleware and software integration concepts. One reason for this situation is the fact that the complexity and variety of systems and applications has grown considerably since then leading to a diversity of approaches. Another reason is clearly the lack of any sound methodology to measures and compare architectural designs. This article tries to address the problem by proposing a comparison and evaluation methodology for robotics software systems. The methodology facilitates an assessment of software systems through experimentation and takes into account the possible operational profile and quality attribute requirements of the robotic application. The approach is validated through the evaluation of three software systems, i.e. GenoM, ORCA2 and GO. The results suggest possible applicability domains of each system and show the efficiency of the approach.