An evaluation of video fidelity metrics

We assess the capability of four metrics (average MSE, average SNR, ANSI parameters, and ITS metric) to determine the fidelity of video sequences. First, we discuss the ideal requirements for a video fidelity metric, in terms of monotonicity, degree of change, and consistent behavior. Then, we construct a series of highly reproducible degraded sequences containing artifacts common to DCT based transform coders, such as H.263, and evaluate the performance of each metric on those sequences. From the resulting data, we determine the accuracy and reliability of each of those metrics. Our analysis can help guide the choice of an appropriate video fidelity metric for evaluating algorithm and architecture choices and tradeoffs.