Enhancing the Credibility of Wireless Network Simulations with Experiment Automation

The last few years have witnessed a growing consensus around the notion that many papers discussing wireless network simulation are plagued by issues that weaken their scientific value. A number of articles have shown evidence of this crisis of credibility and identified many of its causes. In this paper, we show that the methodology flaws in wireless network simulation can be avoided with the use of a framework for experiment automation. We describe the rationale that drove us to develop tools for component-based simulators intending to guide the experimental process from first to last stages. We conclude that a framework that imposes the right constraints on the experimenter can lead to more credible simulation studies. The framework we present helps the construction of consistent models, the definition of model parameters, the design and the execution of experiments, the analysis of output data, and the preparation of data for the dissemination of results that allow experiments to be reproduced.