Unrestricted BIC context tree estimation for not necessarily finite memory processes

Context trees of arbitrary stationary ergodic processes with finite alphabets are considered. Such a process is not necessarily a Markov chain, so the context tree may be of infinite depth. Calculated from a sample of size n, the Bayesian information criterion (BIC) is shown to provide a strongly consistent estimator of the context tree of the process, via minimization over hypothetical context trees, without any restriction on the hypothetical context trees. Strong consistency means that the estimated context tree recovers the true one up to any fixed level K, eventually almost surely as n tends to infinity. This generalizes the previous results, where either the context trees were assumed to be of finite depth or the depth of the hypothetical context trees was bounded by O(log n). Moreover, under some conditions on the process it is also shown that the level K above can grow with n at a specific rate determined by the distribution of the process; thus the BIC estimator can recover the true context tree to larger and larger depths.