Combining perception and knowledge processing for everyday manipulation

This paper describes and discusses the K-COPMAN (Knowledge-enabled Cognitive Perception for Manipulation) system, which enables autonomous robots to generate symbolic representations of perceived objects and scenes and to infer answers to complex queries that require the combination of perception and knowledge processing. Using K-COPMAN, the robot can solve inference tasks such as identifying items that are likely to be missing on a breakfast table. To the programmer K-COPMAN, is presented as a logic programming system that can be queried just like a symbolic knowledge base. Internally, K-COPMAN is realized through a data structure framework together with a library of state-of-the-art perception mechanisms for mobile manipulation in human environments. Key features of K-COPMAN are that it can make a robot environment-aware and that it supports goal-directed as well as passive perceptual processing. K-COPMAN is fully integrated into an autonomous mobile manipulation robot and is realized within the open-source robot library ROS.