Framework for consistent maintenance of geometric data and abstract task-knowledge from range observations

We present a framework for on-line exploration of object attributes from range data designed to include the cognitive aspects for surprise detection. In this framework we introduce a layered representation of the environment that couples the pure geometric 3D representation of the world to the abstract knowledge about the structures in the scene. This knowledge in the higher layer represents a-priori known, task-relevant information about structures in the world like mass, handling properties and grasping points being examples in the case of a manipulation task. The coupling of abstract knowledge to the geometry in the dual layered structure of our map helps to ensure consistency of the representation. The focus of the paper is on data association of dense 3D points from range sensors. We introduce a z-buffered re-projection method as a way to filter outlier information in sensor readings and present our technique for fusion based on the uncertainties in the map representation and the current observation. In contrast to common registration methods, our approach does not store the data as one rigid model but as a set of independent point clusters (foreground) embedded in a 3D point cloud of the supporting structure (background). This allows us to cope with dynamic changes in the world. We register the incoming data not rigidly to the entire map but we update independently the pose of single objects represented in our hierarchical model. The fusion approach combines a local-heuristic with a global-robust procedure and the correspondence search cost of O(nm) is reduced to a set of m sub-searches with linear cost each. The benefit of the re-projection is twofold: it helps speeding up the point matching search by ordering the 3D data according to the manner they might have been captured and making the matching process robust by filtering out outliers and occluded object parts. We present the theoretical framework and we validate our approach on range data from a binocul- r stereo setup.