Prescient mode selection for robust video coding

In standard predictive video coders, intra-mode coding of macroblocks (MBs) provides packet loss resilience, at the cost of reduced compression efficiency. Conventional mode selection algorithms are "greedy" as they focus on minimization of distortion of current MB, while ignoring the effect of this selection on subsequent frames. This paper proposes an algorithm for prescient mode selection: the coding modes of MBs are chosen while taking into account the distortion of subsequent frames. The problem is formulated as one of joint selection of MB coding modes, for a group of pictures, so as to minimize the rate-distortion cost. The straightforward solution based on dynamic programming requires enormous computational complexity. We propose an iterative algorithm which obtains a locally optimal solution at feasible complexity. The total decoder distortion is computed using recursive optimal per-pixel estimate (ROPE) which accurately accounts for the effects of quantization, packet loss, error propagation, and error concealment. Simulation results show consistent improvement over our previous "greedy" ROPE-RD mode selection, and substantial gains over other (non-ROPE) mode selection schemes