Title :
Small file access optimization based on GlusterFS
Author :
Xie Tao ; Liang Alei
Author_Institution :
Sch. of Software Eng., Shanghai Jiao Tong Univ., Shanghai, China
Abstract :
This paper describes a strategy to optimize small file´s reading and writing performance on traditional distributed file system. Traditional distributed file system like GlusterFS stores data within local file system (XFS, EXT3, EXT4, etc.), which shows a significant bottleneck on file metadata lookup. We try to re-design metadata structure by merging small file into large file, thus to reduce size of metadata, so we can store the whole files´ metadata inside main memory. We design and implement the whole strategy on GlusterFS, test results show a great performance optimization on small file operation.
Keywords :
distributed databases; meta data; storage management; table lookup; GlusterFS; data storage; distributed file system; file metadata lookup; local file system; metadata size reduction; metadata structure redesign; small file access optimization; small file reading performance; small file writing performance; Performance evaluation; Reliability; distributed file system; glusterfs; metadata; optimization; small file;
Conference_Titel :
Cloud Computing and Internet of Things (CCIOT), 2014 International Conference on
Print_ISBN :
978-1-4799-4765-2
DOI :
10.1109/CCIOT.2014.7062514