Strongly consistent code-based identification and order estimation for constrained finite-state model classes

Observations are made of data generated by a stationary ergodic finite-alphabet information source according to an unknown statistical model. Two modeling problems, the identification problem and the order estimation problem, are considered. It is required that the given model class in the identification problem and each given model class in the order estimation problem be a constrained finite-state model class, which is a type that includes many model classes of information-theoretic interest. Strongly consistent decision rules are exhibited in both the identification problem and the order estimation problem. The decision rules are code-based in that a model class is chosen based on how well a certain code for that class encodes the observed data. The code used for a model class is based on the maximum likelihood code for that class, and asymptotic code performance is gauged by means of a key property of divergence-rate distance