Prefixes and the entropy rate for long-range sources

The asymptotic a.s.-relation H=lim_n→∞[(nlogn)/(Σ_i=1ⁿ L_iⁿ(X))] is derived for any finite-valued stationary ergodic process X=(X_n, n∈Z) that satisfies a Doeblin-type condition: there exists r⩾1 such that ess_xinf P(X_n+1|x_→∞,n)⩾α>0. Here, H is the entropy rate of the process X, and L_iⁿ(X) is the length of a shortest prefix in X which is initiated at time i and is not repeated among the prefixes initiated at times j, 1⩽i≠J⩽n. The validity of this limiting result was established by Shields in 1989 for i.i.d. processes and also for irreducible aperiodic Markov chains. Under our new condition, we prove that this holds for a wider class of processes, that may have infinite memory