Improving the Performance of Remote I/O Using Asynchronous Primitives

An increasing number of scientific applications need efficient access to large datasets held at remote storage facilities. However, despite the availability of high-speed Internet backbones, the performance penalty of remote I/O is still relatively high compared to local I/O. In this paper we describe different ways in which asynchronous primitives can be used to improve the performance of remote I/O in the grid environment. We have implemented and evaluated three optimization techniques using asynchronous primitives. These primitives have been integrated into SEMPLAR, a high-performance, remote I/O library based on the SDSC storage resource broker. Based on measurements of representative high-performance applications running on three different clusters, we show that different optimization techniques work best for each specific combination of application and platform characteristics. We achieved over 90% overlap between the computation and I/O phase of two applications by using an asynchronous version of remote I/O primitives. We were able to increase the average read and write bandwidth of the ROMIO perf benchmark by 96% and 43% respectively by moving data concurrently over multiple remote connections. Finally, we experienced an improvement of up to 84% in the average write bandwidth when using asynchronous, on-the-fly data compression