Prediction of second-order statistics in motion-compensated video coding

Motion-compensated prediction is widely used in video coding. In current coders, it only predicts the first-order statistics (mean value). However, we show in this paper that the variance of each individual pixel in the residual field can be predicted too, by relating it to the gradient magnitude computed from the previous frame. This relationship leads to designing the residual field coder such that optimal Karhunen-Loeve transforms can be computed for each block. Furthermore, no matter what transform is used, the variance of each individual transform coefficient is known to both ends of the coding process, and hence conditional entropy coding can be implemented