Shannon meets Bellman: Feature based Markovian models for detection and optimization

The goal of this paper is to develop modeling techniques for complex systems for the purposes of control, estimation, and inference: (i) A new class of hidden Markov models is introduced, called the optimal feature prediction (OFP) model. It is similar to the Gaussian mixture model in which the actual marginal distribution is used in place of a Gaussian distribution. This structure leads to simple learning algorithms to find an optimal model. (ii) The OFP model provides a unification of other modeling approaches including the projective methods of Shannon, Mori and Zwanzig, and Chorin, as well as a version of the binning technique for Markov model reduction. (iii) Several general applications are surveyed, including inference and optimal control. Computation of the spectrum, or solutions to dynamic programming equations are possible through a finite dimensional matrix calculation without knowledge of the underlying marginal distribution on which the model is based.