Information-maximizing prefilters for quantization

This work discusses open-loop and closed-loop prediction from an information-theoretic point-of-view. It is shown that the open-loop predictor which minimizes the mean-squared prediction error differs from the filter maximizing the information rate, but that this difference vanishes for high quantizer resolutions. The filter minimizing the mean-squared reconstruction error performs worse for all quantizer resolutions. For the closed-loop predictor, which is shown to be superior only at low quantizer resolutions, the filters maximizing the information rate and minimizing the mean-squared reconstruction error coincide. We illustrate these results with a simple example and discuss similarities with the information-theoretic aspects of principal components analysis and anti-aliasing filtering. Furthermore, we briefly discuss the classical Wiener filter followed by a quantizer.