Rate distortion optimized mode decision in the scalable video coding

In this paper, we discuss how to apply the rate distortion technique to select the optimal mode in the scalable coding. Firstly, we analyze this problem from a general scalable model and point out that the complicated dependencies among the different macroblocks and layers make the original independency assumption no longer a right approximation. Secondly, we propose an EOD function to estimate this dependency and derive a simple formula of this function. We apply the proposed algorithm to the H.26L PFGS, and the experimental results show that the algorithm significantly improves the coding efficiency of the H.26L PFGS. Further studies on how to design the EOD function more accurately is quite interesting and significant.