Length of minimal forbidden words on a stationary ergodic source

An anti-dictionary is in particular useful for data compression, and it consists of minimal forbidden words for a given string. We derive the average length M_n of minimal forbidden words in strings of length n under a stationary ergodic source with entropy H which takes values on a finite alphabet. For the string length n, we prove, log n/M_n = H, in probability, as n rarr infin. We use the Wyner-Ziv result, with respect to connection between entropy and recurrence-time for ergodic processes, to prove the theorem. Its validity is shown by simulation results on a memoryless binary information source.