Efficient File Communication via Deduplication over Networks with Manifest Feedback

We describe a deduplication based file communication system with manifest feedback to save network bandwidth by detecting and not sending duplicate "chunks" of the file already in the procession of the receiver. The manifest feedback mechanism accelerates the file communication process over a network via reducing the overheads associated with the interactive duplication detection and query processes. Test results using a 32 GB real-world data set show that the proposed mechanism achieves over 20% file transmission execution time reduction while avoiding close to 50% of the duplication look-up operations and saving about 60% network bandwidth.