Adaptive CDCLZW algorithm for data compression

With the development of technology, billions of clients visit web servers to surfing on the Internet every day. Insight is a code flow tracing project for web servers. The tracing data in Insight rising with the increasing quantity of web accessing is enormous. JavaScript Object Notation (JSON), a lightweight, text-based, language-independent data interchange format, is used in Insight to transfer data between clients and server. This procedure often contains the transferred code flows with most of data is similar, which can be compressed to save bandwidth. This paper proposes a new adaptive Content Defined Chunking Lempel Ziv Welch (CDCLZW) algorithm for the Insight, to compress JSON data as far as possible and save the network bandwidth. The CDCLZW algorithm uses Template Library and Rabin´s fingerprints to find same data and avoid transferring them between clients and servers. The test environment is developed to investigate the new algorithm, and the results demonstrate that CDCLZW can insure the compression ratio steady and performs better than GNUzip (gzip) and bzip2 algorithms.