Benchmarking Embedded Devices for Broadband Performance Testing

Real time monitoring of broadband performance parameters is critical for estimating the user experience of new broadband services like VoIP, IPTV, Gaming and Video. This information is of interest to service providers themselves for efficient network design and maintenance and government regulatory bodies for analyzing ISPs, regions and national benchmarking. A web-based system TRUEE (Tool for Real-time User Experience Estimation) is a distributed system that incorporates independent modules such as standalone measurement devices installed at customer premises, data centers, test servers and web-clients for remote monitoring and management of the system. The focus of this paper is to discuss the process of benchmarking three commercial embedded devices with PC as reference device representing an end user system for accessing broadband services. This work is part of the ongoing development process of TRUEE. This benchmarking process is of significant importance for making an informed decision on the suitability of an embedded device capable of providing desired accuracy and consistency in estimation of the broadband performance parameters. Based on literature review, online forum reviews and cost analysis three devices based on ARM viz. SheevaPlug, Texas Instrument´s BeagleBoard-xM and Gumstix Overo are selected for benchmarking. Results show that Marvell´s SheevaPlug outperforms the other two devices in accurately measuring the broadband parameters on its network interface.