Experimental analysis of 10Gbps transfers over physical and emulated dedicated connections

Long haul data transfers require the optimization and balancing of performances of host and storage systems as well as network transport. An assessment of such transport methods requires a systematic generation of throughput profiles from measurements collected over different system parameters and connection lengths. We describe host and storage systems to support wide-area I/O transfers at 10 Gbps, and present measurements of memory and disk transfer throughputs over suites of physical and emulated connections of several thousands of miles. The physical connections are limited by the infrastructure and incur significant costs. The emulated connections could be of arbitrary lengths at significantly lower costs but only approximate the physical connections. We present a differential regression method to estimate the differences between the performance profiles of physical and emulated connections, and then to estimate “physical” profiles from emulated measurements. We present a systematic analysis of wide-area memory and disk transfer throughput measurements, and establish that robust estimates of physical profiles can be generated using much less expensive emulated connections.