Sensor fusion and planning with perception-action network

An architecture of intelligent robotic systems, referred to here as perception-action net (PAN), is presented. Connecting sensing and action in real-time, PAN automatically synthesizes goal-oriented behaviors under uncertainties, errors, and faults, through task monitoring and replanning. PAN is composed of the perception and action nets interconnected in closed loops. The perception net connects features of various levels of abstraction or logical sensors in hierarchy. The net is capable of self-calibrating itself by maintaining the consistency of logical sensors based on the forward propagation of sensor outputs and uncertainties as well as based on the backward propagation of errors from constraints. The action net consists of a hierarchy of state transition networks of multiresolution time scales. The net embeds all the feasible system behaviors in various levels of abstraction, such that the system can replan and control its behaviors towards the set goals under errors and faults. The application of PAN to planetary robotic sampling is shown