Non-integer bit estimation for rate-distortion optimized video coding

Many modern video encoders use rate-distortion optimization (RDO) methods for choosing coding parameters. This approach involves estimation of rate and distortion for different coding configurations. Intuitively, better performance can be achieved with more accurate rate and distortion assessments. This paper proposes improved rate computation by estimating a non-integer number of bits for arithmetic coding of syntax elements. This is applied to optimization of the H.264 codec in arithmetic coding mode and promising results are achieved.