Real versus realistically rendered scenes for optical flow evaluation

In this paper we evaluate the effects of physically plausible renderings of ground truth sequences for optical flow estimation, as it seems to be a common belief that real world effects such as specular highlights, shadows, etc. cannot easily be reproduced synthetically while their influence on optical flow is hard to classify. Therefore, a real sequence with shadows and specular reflections was recorded and then reconstructed using raytracing renderers for varied lighting conditions and material properties. Due to the complexity of the task we limited the calculation of the optical flow to a single algorithm with a single set of parameters. The resulting endpoint errors are analyzed with respect to their mean and spatial distribution. We compare the optical flow estimates of the real scene with the synthetic scenes and find that the results are similar in mean endpoint error but not in its spatial distribution. Our results indicate the need for further studies focusing on the usefulness of synthetic ground truth sequences.