3DNBS: A Data De-duplication Disk-Based Network Backup System

Traditionally, backup and archiving have been performed on tapes. With the rapid advances in disk storage technology witnessed in recent years, it becomes practical to use disks other than tape libraries as backend storage device for a backup system. For such a disk-based system, storage space efficiency is essential. Since traditional backup method cannot eliminate redundancies during backup, a new data de-duplication backup technique should be developed to provide more efficient data storage at the system. This paper describes the design and performance evaluation of a data de-duplication disk-based network backup system,called 3DNBS. 3DNBS breaks files into variable sized chunks using content-defined chunking (CDC) for the purpose of duplication detection. Chunks are indexed and addressed by hashing their content, which leads to intrinsically single instance storage. Experimental results show that in comparison with traditional backup method such as Bacula, 3DNBS presents dramatic reduction in required storage space on various workloads. By eliminating duplicated data, 3DNBS also reduces the size of data to be transmitted, hence reducing time to perform backup in a bandwidth constraint environment.