Systems considerations in real time video QoE assessment

We consider several key questions in the design of a real time video quality assessment system. How frequently can we generate subjective video quality ratings with some degree of accuracy? How often should we sample the data? How do we weigh the need to consolidate data collection (arguing for fewer, less frequent data points) with the need to monitor video quality in real time (arguing for more frequent data points)? What are the timing requirements for such a system, both in training the system and in assigning ratings to videos? Our results demonstrate that we can achieve accurate video quality ratings by using small portions of the video and a frequent sampling rate, and that in the worst case the system can be trained in ten minutes.