Traps and pitfalls of using contact traces in performance studies of opportunistic networks

Contact-based simulations are a very popular tool for the analysis of opportunistic networks. They are used for evaluation of networking metrics, for quantifying the effects of infrastructure and for the design of forwarding strategies. However, little evidence exists that the results of such simulations accurately describe the performance of opportunistic networks, as they commonly ignore some important factors (like limited transmission bandwidth) or they rely on assumptions such as infinite user cache sizes. In order to evaluate this issue, we design a testbed with a real application and real users; we collect application data in addition to the contact traces and compare measured performance to the results of the contact-based simulations. We find that contact-based simulations significantly overestimate delivery ratio, while the captured delay tends to be 2-3 times lower than the experimentally obtained delay. We show that assuming infinite cache sizes leads to misinterpretation of the effects of backbone on an opportunistic network. Finally, we show that contact traces can be used to analytically estimate the delivery ratios and the impact of backbone, through the dependency between a user centrality measure and her delivery ratio.