Empirical processes and typical sequences

This paper proposes a new notion of a typical sequence over an abstract alphabet based on approximation of memoryless sources by empirical distributions, uniformly over a class of measurable “test functions.” In the finite-alphabet case, we can take all uniformly bounded functions and recover the usual notion of typicality under total variation distance. For a general alphabet, this function class is too large, and must be restricted. We develop our notion of typicality with respect to any Glivenko-Cantelli function class (which admits a Uniform Law of Large Numbers) and demonstrate its power by deriving fundamental limits on achievable rates in several settings that can be reduced to uniform approximation of general-alphabet memoryless sources with respect to a suitable function class.