MPEG coding performance improvement by jointly optimizing coding mode decisions and rate control

This paper presents a new algorithm for determining the optimal MPEG coding strategy in terms of the selection of macroblock coding modes and quantizer scales. In the algorithm proposed in the test model the rate control operates independently from the coding mode selection for each macroblock. The coding mode is decided based only upon the energy of predictive residues. Actually, the two processes of coding mode decision and rate control are intimately related to each other and should be determined jointly in order to achieve optimal coding performance. We formulate the constrained optimization problem and present solutions based upon rate-distortion characteristics, or R(D) curves, for all the macroblocks that compose the picture being coded. Distortion for the entire picture is assumed to be decomposable and expressible as a function of individual macroblock distortions, with this being the objective function to minimize. The determination of the optimal solution is complicated by the MPEG differential encoding of motion vectors and DC coefficients, which introduce dependencies that carry over from macroblock to macroblock for a duration equal to the slice length. As an approximation, a near optimum greedy algorithm is proposed. Once the upper bound in performance is calculated, it can be used to assess how well practical suboptimum methods perform. Finally, such a practical suboptimum algorithm is proposed and evaluated