Perceptual evaluation of image denoising algorithms

Image denoising has been an active research topic in the past decades for its broad real-world applications, but surprisingly little work has been dedicated to the quality assessment of de-noised images. In this work, we first build a database that contains noisy images at different noise levels and denoised images created by both classical and state-of-the-art denoising algorithms. We then carry out a subjective experiment using a multi-stimulus ranking approach to evaluate and compare the quality of the denoised images. Data analysis shows that there are both considerable agreement and significant variations between human subjects on their opinions of denoised images. Our results also show that state-of-the-art objective image quality models only moderately correlate with subjective opinions, and further investigations that involve both structural fidelity and naturalness measures are desirable in future development of advanced objective models.