SDVC: A Scalable Deduplication Cluster for Virtual Machine Images in Cloud

Nowadays, while the storage requirement of virtual machine images generated in cloud infrastructures can be potentially reduced by the deduplication, considering their scale and intensity, the deduplication cluster is demanded. Therefore, in this paper we present SDVC, a scalable deduplication cluster for virtual machine images in cloud. SDVC offers both vertical and horizontal scalability. The horizontal scalability is supported by a three-party distributed infrastructure and a hash allocation algorithm. Meanwhile, categorized chunk tracer and buffer capture hot data. Furthermore, SDVC is vertical scalable by setting a suitable hot chunk buffer in virtual machine servers according to their resource usage, reducing chunk searching operations and relieving the workloads on dedup servers. Our experimental results based on a small scale cluster show that the deduplication throughput achieves up to 80% increase with the number of Dedup servers. Furthermore, only hundreds of Kbytes of categoried hot chunk buffer can provide almost 100% performance improvement.