QDFS: A quality-aware distributed file storage service based on HDFS

Author

Song Guang-hua ; Chuai Jun-na ; Yang Bo-wei ; Zheng Yao

Author_Institution

Center for Eng. & Sci. Comput., Zhejiang Univ., Hangzhou, China

Volume

2

fYear

2011

fDate

10-12 June 2011

Firstpage

203

Lastpage

207

Abstract

On the basis of the Hadoop distributed file system (HDFS), this paper presents the design and implementation of QDFS, a distributed file storage service system that employs a backup policy based on recovery volumes and a quality-aware data distribution strategy. The experimental results show that QDFS increases the storage utilization ratio and improves the storage performance by being aware of the quality of service of the DataNodes. The improvements of QDFS compared with HDFS make the Hadoop distributed computing model more suitable to unstable wide area network environments.

Keywords

distributed databases; quality of service; storage management; wide area networks; DataNodes; HDFS; Hadoop distributed computing model; Hadoop distributed file system; QDFS; backup policy; distributed file storage service system; quality of service; quality-aware data distribution strategy; quality-aware distributed file storage service; storage utilization ratio; wide area network environment; Aerospace electronics; Bandwidth; Computational modeling; Computers; Distributed databases; Internet; Quality of service; HDFS; cloud storage; hadoop; quality-aware; redundant backup;

fLanguage

English

Publisher

ieee

Conference_Titel

Computer Science and Automation Engineering (CSAE), 2011 IEEE International Conference on

Conference_Location

Shanghai

Print_ISBN

978-1-4244-8727-1

Type

conf

DOI

10.1109/CSAE.2011.5952454

Filename

5952454