At home in the lab: Assessing audiovisual quality of HTTP-based adaptive streaming with an immersive test paradigm

In this paper, we assess audiovisual quality of HTTP Adaptive Streaming using a novel subjective test design. The goal of this test was to systematically study the impact of both quality variations and stalling events on remembered quality. To gather more ecologically valid results, we wanted to reach a degree of test subject engagement and attention closer to real-life video viewing than what is achieved in traditional lab tests. To this aim, we used a novel test design method: the “immersive” paradigm, where subjects never see the same source stimulus twice. A total of 66 source clips of one minute length have been shown, selected from online video services. Together with qualitative results obtained from questionnaires, we can confirm previously reported effects, such as the impact of quality switching frequency. We also present new findings on the interaction of stalling events and quality drops. Finally, the contribution highlights that a long duration test of over one hour is feasible using the immersive paradigm while keeping subjects entertained.