New concepts in nonlinear infinite-horizon stochastic estimation and control: The finite element case

A finite probabilistic system (FPS) is a discrete-time controlled stochastic process having finite input, output, and (internal) state sets. (A partially-observed Markov decision process is an example of an FPS). It may be viewed as the simplest formulation of a nonlinear estimation and control problem. Under conditions similar to observability and controllability in linear systems, the problem of selecting inputs, on the basis of past inputs and outputs (with perfect recall), so as to maximize a time-averaged expected reward, is shown to be meaningful as the horizon increases without bound or as a discount approaches unity: an optimal strategy exists; it may be realized by, a (strategy-independent) state estimator along with a stationary policy on the state estimate; and its performance does not depend on the initial state of information. Dual control aspects of the problem, and potential extention of the results to more general systems are briefly discussed.