Title :
HDFSbench: Understanding the Efficiency and Bottlenecks of Cloud File Systems
Author :
Jinquan Dai ; Tao Xie ; Shengsheng Huang ; Jie Huang
Abstract :
We have conducted intensive experiments on an in-house Hadoop cluster using HDFSbench (a file system benchmark tool we build for HDFS). Our experimental results provide valuable insights into the performance characteristics (e.g., general efficiency and potential bottlenecks) of cloud file systems for different application usages (e.g., MapReduce and Bigtable access patterns), and on how these traits change with new storage technologies (e.g., SSD vs. HDD).
Keywords :
benchmark testing; cloud computing; file organisation; storage media; HDFSbench; Hadoop Distributed File System; Hadoop cluster; benchmark tool; cloud file systems; storage technologies; Aggregates; Bandwidth; Benchmark testing; Clouds; Data models; Distributed databases; File systems; Big Data; HDFS; Hadoop;
Conference_Titel :
Open Cirrus Summit (OCS), 2012 Seventh
Conference_Location :
Beijing
DOI :
10.1109/OCS.2012.31
