A Sketch Interface for Robust and Natural Robot Control

In this paper, a novel approach for commanding mobile robots using a probabilistic multistroke sketch interface is presented. Drawing from prior work in handwriting recognition, sketches are modeled as a variable duration hidden Markov model, where the distributions on the states and transitions are learned from training data. A forward search algorithm is used to find the most likely sketch given the observations on the strokes, interstrokes, and gestures. A heuristic is implemented to discourage breadth-first search behavior, and is shown to greatly reduce computation time while sacrificing little accuracy. To avoid recognition errors, the recognized sketch is displayed to the user for confirmation; a rejection prompts the algorithm to search for and display the next most likely sketch. Upon confirmation of the recognized sketch, the robot executes the appropriate behaviors. A set of experiments was conducted in which operators controlled a single mobile robot in an indoor search-and-identify mission. Operators performed two missions using the proposed sketch interface and two missions using a more conventional point-and-click interface. On average, missions conducted using sketch control were performed as well as those using the point-and-click interface, and results from user surveys indicate that more operators preferred using sketch control.