Design and logistics in formal subjective test the MPEG case

This paper describes the strategies adopted in the design and logistic organization of formal subjective assessment experiments of big dimensions. The selection of the logistic solutions and the test design are strongly influenced by the importance of the experiment and by the fact that the results of the test are going to be one of the aspects that will be taken into account in the selection of new technologies to be standardized. The case described in this paper is related to the recent formal subjective assessment tests conducted by MPEG to promote the standardization of new technologies in the area of 2D and 3D video compression and synthesis. The test considered were extremely challenging due to the very wide participation and, in the case of 3D, to the novelty of the testing experiment never done before for 3D content.