Depth map enhanced macroblock partitioning for H.264 video coding of computer graphics content

In this paper, we present a method to speed up video encoding of GPU rendered scenes. Modern video codecs, like H.264/AVC, are based on motion compensation and support partitioning of macroblocks, e.g. 16×16, 16×8, 8×8, 8×4 etc. In general, encoders use expensive search methods to determine suitable motion vectors and compare the rate-distortion score for possible macroblock partitionings, which results in high computational encoder load. We present a method to accelerate this process for the case of streaming graphical output of unmodified commercially available 3D games which use a Skybox or Skydome rendering technique. For rendered images, usually additional information from the render context of OpenGL resp. DirectX is available which helps in the encoding process. By incorporating the depth map from the graphics board, such regions can be uniquely identified. By adapting the macroblock partitioning accordingly, the computationally expensive search methods can often be avoided. Further reduction of encoding load is achieved by additionally capturing the projection matrices during the Skybox rending and using them to directly calculate a motion vector which is usually the result of expensive search methods. In experiential results, we demonstrate the reduced computational encoder load.