Interactive command building and sequencing for supervised autonomy

An operator interface for interactive command building and sequencing for supervised autonomy is described. The system is the ground operator control station of a local-remote telerobotics system being developed for time-delayed remote-control of Space Station Freedom robots. The system provides stereo graphics overlay on video with interactive update of the environment. The operator selects objects in the environment with which to interact, and skills to specify the task to be performed, such as grasping a module, or opening a door. The information needed by the skill to operate on a specific object is stored in a knowledge base. In more complex cases, such as inserting a grasped module into a receptacle or using a tool on an object, the parameterization is generated by the skill querying the knowledge base for both the tool being held and the object on which the operation is performed