Characterizing throughput bottlenecks for secure GridFTP transfers

GridFTP is the de facto standard for bulk data movement in distributed science environments. It extends the legacy FTP to provide strong security, reliability, and high performance. GridFTP, like FTP, is a two-channel protocol-the control channel is used for sending commands and responses, and the data channel is used for transferring the actual data. The control channel is encrypted and integrity protected by default. The data channel is authenticated by default. Encryption and integrity protection are both supported on the data channel but are not enabled by default because of their high CPU cost and low data transfer rates. In this paper, we present an extensive experimental study on the performance implications of enabling integrity protection and encryption on the data channel. We show that in a vast number of cases involving the use of nonthreaded Globus GridFTP servers on multicore systems, throughputs of secure transfers are not comparable to those of nonencrypted and nonintegrity-protected transfers because of an inefficient use of available processors. However, in cases where a strong desire for higher security levels permits larger expenditures in processing, integrity protection and sometimes even crypto-graphic confidentiality can be provided without having to suffer a decline in throughput. We show that this can be accomplished through threaded Globus GridFTP server instances configured with appropriately chosen parallelism and concurrency, allowing for a more effective use of available system resources.