An end-to-end approach for optimal mode selection in Internet video communication: theory and application

Rate-distortion (R-D) optimized mode selection is a fundamental problem for video communication over packet-switched networks. The classical R-D optimized mode selection only considers quantization distortion at the source. Such an approach is unable to achieve global optimality under the error-prone environment since it does not consider the packetization behavior at the source, the transport path characteristics, and receiver behavior. This paper presents an end-to-end approach to generalize the classical theory of R-D optimized mode selection for point-to-point video communication. We introduce a notion of global distortion by taking into consideration both the path characteristics (i.e., packet loss) and the receiver behavior (i.e., the error concealment scheme), in addition to the source behavior (i.e., quantization distortion and packetization). We derive, for the first time, a set of accurate global distortion metrics for any packetization scheme. Equipped with the global distortion metrics, we design an R-D optimized mode selection algorithm to provide the best tradeoff between compression efficiency and error resilience. The theory developed in this paper is general and is applicable to many video coding standards, including H.261/263 and MPEG-1/2/4. As an application, we integrate our theory with point-to-point MPEG-4 video conferencing over the Internet, where a feedback mechanism is employed to convey the path characteristics (estimated at the receiver) and receiver behavior (error concealment scheme) to the source. Simulation results are discussed.