Automatic Testing and Measurement of QoE in IPTV Using Image and Video Comparison

Orange Labs Spain, has developed a proof of concept aimed at exploring the possibility of automating the tests required by new evolutions of the IPTV service, and thus shortening the lifecycle and reducing the overall development costs. It´s based on an automaton that interacts with the set-top box just by emulating a human being (or tester): sending IR remote control commands and analyzing the video output, as a set of image comparisons against a valid reference. The analysis unveiled a future applicability of the same concept in objectively measuring the QoE (quality of experience). Although there are techniques and measuring equipment in the industry, they often mix QoS (quality of services) with QoE, and causes with effects. In our analysis, we propose to clearly separate QoE (as an effect) from QoS (as possible causes), and also to split the contributions to QoE in those that occur in the preparation of the content (such as the encoding of VoDs (video on demand) in backoffice), and those that take place at the moment of content delivery itself. Finally, we consider that the customer environment is an element that needs to be considered as a critical part the loop of quality. As a conclusion, we consider that managing and control the QoE is the future to improve the customer experience.