Give me a hand — How users ask a robotic arm for help with gestures

A task that requires two hands to perform such as soldering usually needs additional tools for holding (e.g. a cable) or adding (e.g. solder) an object to a specific position. A robotic manipulator or robotic arm is one of the solution for this requirement. When gesture is selected as a method for controlling a robot, characteristics of gestures are needed for designing and developing a gesture recognition system. With this requirement, we conducted an experiment to obtain a set of user-defined gestures in the soldering task to find out properties and patterns of a gesture for future development of our research. 152 gestures were collected from 19 participants by presenting the “effect” of the gesture (robotic arm movement), and then asking the participants to perform its “cause” (a user-defined gesture). The analyzed data shows that hands are the most used body parts even they are occupied by the task, that one-hand and two-hands gestures were used interchangeably by the participants, that the majority of the participants performed reversible gestures for reversible movements, and that the participants were expecting for better recognition performance on an easier to plan gesture. Our finding can be useful as a guideline for creating gesture set and system for controlling robotic arms based on natural behavior of users.