Divergence of information-criterion based Markov order estimators for infinite memory processes

For finite-alphabet stationary ergodic processes with infinite memory, Markov order estimators that optimize an information criterion over the candidate orders based on a sample of size n are investigated. Three familiar information criteria are considered: the Bayesian information criterion (BIC) with generalized penalty term yielding the penalized maximum likelihood (PML), and the normalized maximum likelihood (NML) and the Krichevsky-Trofimov (KT) code lengths. A bound on the probability that the estimated order is greater than some order is obtained under the assumption that the process is weakly non-null and α-summable. This gives an O(log n) upper bound on the estimated order eventually almost surely as n → +∞. Moreover, a bound on the probability that the estimated order is less than some order is obtained if the decay of the continuity rate of the weakly non-null process is in some exponential range. This implies that then the estimated order attains the O(log n) divergence rate eventually almost surely as n → +∞.