Clustering metadata servers for extremely scalable and reliable services

LakeFS[1] is a cluster file system designed for high scalable and reliable storage service. It provides excellent scalability and availability on data I/O operations. However, it lacks the ability to scale up the metadata operations, which makes the metadata server (MDS) as a single point of bottleneck especially when full scan of metadata is needed. In this paper, we present a way to cluster MDSs to solve the bottleneck problem. We also change the file system utilities to exploit the cluster fully. Experimental results show that the throughput of metadata operation is enhanced linearly as the number of MDS participating in the cluster increases.