Grammar-based lossless universal refinement source coding

A sequence y=(y₁,...,y_n) is said to be a coarsening of a given finite-alphabet source sequence x=(x₁,...,x_n) if, for some function φ, y_i=φ(x_i) (i=1,...,n). In lossless refinement source coding, it is assumed that the decoder already possesses a coarsening y of a given source sequence x. It is the job of the lossless refinement source encoder to furnish the decoder with a binary codeword B(x|y) which the decoder can employ in combination with y to obtain x. We present a natural grammar-based approach for finding the binary codeword B(x|y) in two steps. In the first step of the grammar-based approach, the encoder furnishes the decoder with O(√nlog₂n) code bits at the beginning of B(x|y) which tell the decoder how to build a context-free grammar G_y which represents y. The encoder possesses a context-free grammar G_x which represents x; in the second step of the grammar-based approach, the encoder furnishes the decoder with code bits in the rest of B(x|y) which tell the decoder how to build G_x from G_y. We prove that our grammar-based lossless refinement source coding scheme is universal in the sense that its maximal redundancy per sample is O(1/log₂n) for n source samples, with respect to any finite-state lossless refinement source coding scheme. As a by-product, we provide a useful notion of the conditional entropy H(G_x|G_y) of the grammar G_x given the grammar G_y, which is approximately equal to the length of the codeword B(x|y).