Open Profiling of Quality: AMixedMethod Approach to UnderstandingMultimodal Quality Perception

To quantify the excellence of multimedia quality, subjective ev aluation experiments are conducted. In these experiments, thetradition of quantitative assessment is the most dominating, but it disregards the understanding of participants’ interpretations,descriptions, and the evaluation criteria of quality. The goal of t his paper is to present a new multimedia quality evaluation methodcalled Open Profiling of Quality (OPQ) as a tool for building a deeper understanding on subjective quality. OPQ is a mixedmethod combining a conventional quantitative psychoperceptual eva luation and qualitative descriptive quality evaluation basedon the individual’s own vocabulary. OPQ is targeted for na¨ıve participants applicable to experiments with heterogeneous andmultimodal stimulus material. The paper presents the theoretical basis of the development of OPQ and overviews the methods foraudiovisual quality research. We present three extensive quality evaluation studies where OPQ has been used with 120 participants.Finally, we conclude further recommendations of use of the method in quality evaluation research.