On the entropy rate of pattern processes

Recent work by Orlitsky et al. (2004) has motivated the study of pattern sequences and their compressibility properties. Emphasis in this recent line of work has been on compressing pattern sequences under uncertainty in the source that has generated them, thus focusing on universal schemes and their redundancy. Our interest in this work is in the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for i.i.d. processes over arbitrary alphabets, stationary and ergodic: processes over discrete alphabets, as well as more general processes that can be represented as the output of an additive white-noise channel. For cases where the entropy rate of the pattern process is infinite, we characterize the possible growth rate of the block entropy.