Sketch-based robot programming

Robots are rapidly becoming a part of everyday life and have now moved from industrial environments to household, medical and entertainment applications. In order to make full use of robots new interfaces need to be developed, which allow inexperienced human users to instruct (program) robots, without having to understand programming and the underlying electronics and mechanics. In this paper we present a novel sketch-based interface for robot programming. We have identified applications which are difficult to represent algorithmically, but can be easily represented with sketch input. We then define a range of sketch impressions allowing the user to define a wide range of behaviours within these application domains. Our system uses a Pioneer robot with an arm and a fixed overhead camera. The user sketches into the camera view and the sketch input is interpreted, mapped into the 3D domain, and translated into robot interactions. Current applications include specification of robot paths and obstacles, covering regions (e.g., patrolling in security applications or seed sowing in agricultural applications), and directing the robot arm, e.g., to pick up objects. A user evaluation of the system demonstrates that the interface is intuitive and, with the exceptions of controlling the arm, all interactions are perceived as easy to perform.