A belief-based architecture for scene analysis: From sensorimotor features to knowledge and ontology

Based on the neurobiological and cognitive principles of human information processing, we develop a system for the automatic visual identification and exploration of scenes. The system architecture consists of three layers: a bottom-up feature extraction stage, a top-down object identification stage and knowledge from a domain ontology for scene analysis. The uncertainty in the latter two stages is managed by Dempster–Shafer belief measures. The system sequentially selects “informative” image regions, identifies the local structure in these regions, and uses this information for drawing efficient conclusions about an object in the scene. The selection process involves low-level, bottom-up processes for sensory feature extraction, and cognitive top-down processes for the generation of active motor commands that control the positioning of the sensors towards the most informative regions. Both processing levels have to deal with uncertain data, and have to take into account learned statistical knowledge. For bottom-up feature extraction this is achieved by integrating a nonlinear filtering stage modeled after the neural computations performed in the early stages of the visual system. The top-down cognitive reasoning strategy operates in an adaptive fashion on a belief distribution. The resulting object hypotheses in combination with knowledge from the domain ontology in the third layer are used for generating a scene hypothesis.