Comparing apples and oranges: Subjective quality assessment of streamed video with different types of distortion

Video quality assessment is essential for the performance analysis of visual communication applications. Objective metrics can be used to estimate the relative quality differences, but they typically give reliable results only if the compared videos contain similar type of quality distortion. However, video compression typically produces different kinds of visual artifacts than transmission errors. In this paper, we propose a novel subjective quality assessment method that is suitable for comparing different types of quality distortions. The proposed method has been used to evaluate how well PSNR estimates the relative subjective quality levels for content with different types of quality distortions. Our conclusion is that PSNR is not a reliable metric for assessing the co-impact of compression artifacts and transmission errors on the subjective quality.