A Two-Layered Replica Management Method

Replica management method has been widely used in distributed file system to improve parallelism and reliability. Traditional replica management method is synchronous replication management (SRM) which updates replicas synchronously. Because of synchronous updating, it is difficult to provide high throughput and low access latency. If the system achieves SRM in the memory hierarchy, it will take a lot of memory space. In order to improve write performance, we design a Two Layered Replica Management Method (TLRMM) based on memory and disk. It can be used for disk-based or memory-based data storage system. TLRMM maintains 3 replicas for every chunk and store one in memory. Using asynchronous update, it can improve the system´s I/O performance significantly. We integrated TLRMM in Carrier which is a distributed file system designed by Tsinghua University, and use version number to ensure replica consistency. Furthermore, in order to ensure the reliability of Carrier, we design a replica recovery strategy to solve the failure of single point. We have implemented and evaluated the integrated prototype system. The experimental results show that TLRMM can deliver high performance on distributed file system. Compared with SRM, write throughput of TLRMM increased by a factor of 1.62~2.05, without read performance degradation.