DocumentCode :
1919857
Title :
Poster: MemzNet: Memory-Mapped Zero-Copy Network Channel for Moving Large Datasets over 100Gbps Networks
Author :
Balman, Mehmet
fYear :
2012
fDate :
10-16 Nov. 2012
Firstpage :
1513
Lastpage :
1513
Abstract :
High-bandwidth networks are poised to provide new opportunities in tackling large data challenges in today´s scientific applications. However, increasing the bandwidth is not sufficient by itself; we need careful evaluation of future high-bandwidth networks from the applications´ perspective. We have experimented with current state-of-the-art data movement tools, and realized that file-centric data transfer protocols do not perform well with managing the transfer of many small files in high-bandwidth networks, even when using parallel streams or concurrent transfers. We require enhancements in current middleware tools to take advantage of future networking frameworks. To improve performance and efficiency, we develop an experimental prototype, called MemzNet: Memory-mapped Zero-copy Network Channel, which uses a block-based data movement method in moving large scientific datasets. We have implemented MemzNet that takes the approach of aggregating files into blocks and providing dynamic data channel management. In this work, we present our initial results in 100Gbps networks.
Keywords :
100Gbps networks; data-intensive distributed computing; high-speed networks; large scale data movement;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:
Conference_Location :
Salt Lake City, UT
Print_ISBN :
978-1-4673-6218-4
Type :
conf
DOI :
10.1109/SC.Companion.2012.295
Filename :
6496078
Link To Document :
بازگشت