An information theoretic approach to dynamical systems modeling and identification

The identification and modeling of dynamical systems in a practical situation, where the model set under consideration does not necessarily include the observed system, are treated. A measure of the relevant information in a sequence of observations is shown to possess useful properties, such as the metric property on the parameter set. It is then shown that maximum likelihood and related Bayesian identification procedures converge to a model in the model set, which is closest to the actual system generating the observations in the information distance measure. The convergence analysis is restricted for simplicity to finite sets of models. The analysis naturally suggests methods for approximating a high-order system by a low-order model and for selecting a representative model from a given model set, applicable to infinite and even noncompact model sets.