Near-lossless compression of large alphabet sources

We study universal compression of independent and identically distributed sources over large alphabets using fixed-rate codes. To model large alphabets, we use sequences of discrete alphabets that increase in size with the blocklength. We show that universal compression is possible using deterministic codes provided that the alphabet growth is sub-linear in the blocklength. For linear alphabet growth, we show that universal compression is not possible, even if the use of randomized encoders and decoders is permitted. However, if only the decoder is provided with the source distribution, then randomized universal coding is always possible for any growth rate. For the non-universal case in which the goal is to compress a source generated by a known sequence of distributions, we show that compression at the entropy of the source sequence is possible if and only if the ratio of the square logarithm of the alphabet size to the blocklength goes to zero.