A Modified Rate-Distortion Optimisation Strategy for Hybrid Wavelet Video Coding

Rate-distortion optimisation (RDO) techniques are applied in video coding in order to provide optimum trade-offs between bit rate and quality, by minimising a metric combining measures of entropy and distortion. In hybrid wavelet video coding, inter as well as intra frames are wavelet transformed and encoded. Applying a single quantiser per subband does not allow for local adaption, and with the MSE metric for distortion, causes quality degradation as motion estimation failures are highly localised and poorly captured by the metric. This paper considers a fourth-power metric for measuring quantisation distortion within a subband, together with modifications to RDO motion estimation to reduce motion estimation failures. These techniques improve both subjective and objective performance without requiring multiple quantisers per subband. The approach has been applied to the open source Dirac video codec