AMP: An Affinity-Based Metadata Prefetching Scheme in Large-Scale Distributed Storage Systems

Author

Lin, Lin ; Li, Xuemin ; Jiang, Hong ; Zhu, Yifeng ; Tian, Lei

Author_Institution

Dept. of Comput. Sci. & Eng., Univ. of Nebraska-Lincoln, Lincoln, NE

fYear

2008

Firstpage

459

Lastpage

466

Abstract

Prefetching is an effective technique for improving file access performance, which can significantly reduce access latency for I/O systems. In distributed storage systems, prefetching for metadata files is critical for the overall system performance. In this paper, an affinity-based metadata prefetching (AMP) scheme is proposed for metadata servers in large-scale distributed storage systems to provide aggressive metadata prefetching. Through mining useful information about metadata accesses from past history, AMP can discover metadata file affinities accurately and intelligently for prefetching. Compared with LRU and some of the latest file prefetching algorithms such as Nexus and C-Miner, our trace-driven simulations show that AMP can improve buffer cache hit rates by up to 12%, 4.5% and 4% respectively, while reduce the average response time by up to 60%, 12% and 8%, respectively.

Keywords

data mining; distributed memory systems; meta data; storage management; affinity-based metadata prefetching scheme; buffer cache; data mining; large-scale distributed storage system; Computer science; Data mining; Delay; Distributed computing; File systems; Grid computing; History; Large-scale systems; Prefetching; Scalability; Prefetching; data mining; distributed storage; metadata;

fLanguage

English

Publisher

ieee

Conference_Titel

Cluster Computing and the Grid, 2008. CCGRID '08. 8th IEEE International Symposium on

Conference_Location

Lyon

Print_ISBN

978-0-7695-3156-4

Electronic_ISBN

978-0-7695-3156-4

Type

conf

DOI

10.1109/CCGRID.2008.117

Filename

4534250