Journey through the crowd: Best practices and recommendations for crowdsourced QoE

Crowdsourcing-based QoE testing is easy to implement and gives quick results at low costs; as a result crowdsourced QoE data is nowadays popular. However, there are common mistakes and pitfalls which are to be kept in mind for ensuring the reliability of such data. In this paper we summarize best practices and recommendations learned through a 4 year journey in Qualinet which have been collected in the Qualinet recommendation paper on crowdsourcing [1].