A test framework for large-scale message broker system for consumer devices

This paper presents a test framework for a large-scale message broker system for consumer devices, where communication is between devices and goes through a message broker. In testing such a system, administering tests is a burdensome task, because a tester has to design and operate a test application managing multiple connections and to validate complicated traffic patterns between clients via a broker. In addition, a tester has to validate service availability when numerous devices use the system and some message brokers have gone down or the system is scaled-out/in. In such an eventuality, a message broker in operation is removed from the system or a new message broker is added to the system. In our framework, a tester can write a test scenario to validate message transfers between devices and service availability in the case of changes in the system structure on the fly. In a case study, we implement a prototype framework on Apache JMeter for remote control system for home appliances. We evaluate the target system with our framework in several test scenarios and confirm service availability when one million devices use the system and when scale-out/in occurs.