Efficient File Communication via Deduplication over Networks with Manifest Feedback

Author

Bing Zhou ; Jiangtao Wen

Author_Institution

Dept. of Comput. Sci. & Technol., Tsinghua Unviersity, Beijing, China

Volume

18

Issue

1

fYear

2014

fDate

Jan-14

Firstpage

94

Lastpage

97

Abstract

We describe a deduplication based file communication system with manifest feedback to save network bandwidth by detecting and not sending duplicate "chunks" of the file already in the procession of the receiver. The manifest feedback mechanism accelerates the file communication process over a network via reducing the overheads associated with the interactive duplication detection and query processes. Test results using a 32 GB real-world data set show that the proposed mechanism achieves over 20% file transmission execution time reduction while avoiding close to 50% of the duplication look-up operations and saving about 60% network bandwidth.

Keywords

data communication; feedback; query processing; receivers; bit rate 32 Gbit/s; data deduplication; deduplication based file communication system; duplication look-up operations; efficient file communication; feedback mechanism; file communication process; file transmission execution time reduction; interactive duplication detection; manifest feedback; network bandwidth; query processes; receiver; Acceleration; Bandwidth; Data communication; Protocols; Random access memory; Receivers; Throughput; File communication; data deduplication; manifest feedback;

fLanguage

English

Journal_Title

Communications Letters, IEEE

Publisher

ieee

ISSN

1089-7798

Type

jour

DOI

10.1109/LCOMM.2013.111313.131808

Filename

6676779