Exploring discrete approaches to lossy compression schemes for natural image patches

Optimal compressions in a rate-distortion sense are usually discrete random variables, so clever discretizations of natural images might be key to developing better compression schemes. A new image compression method achieved good perceptual coding performance by using as primitives memories of a Hopfield network trained on discretized natural images. Here we explore why Hopfield network fixed-points are good lossy perceptual features even though the implied generative model (a second-order Lenz-Ising model) does not provide a state-of-the-art match to the true probability distribution of discretized natural images. Even so, we demonstrate that this deterministic coding scheme can achieve near-optimality by comparing with the rate-distortion function for discretized natural image patches.