A log data transmission algorithm based on file splitting and cache matching

The log file for recording events in a cloud platform is maintained by a computer server and generally contains large amounts of data. As networks and transmission are becoming increasingly important for computational science, we are working on proposing a log file transmission algorithm. This new algorithm focuses on optimizing the performance of log file transmission, which is independent of transport protocols. Our approach includes two components: the file splitting and buffer match. The file splitting performs file compression and redundancy elimination, and the buffer memory match avoids transferring the redundant data. Experimental results verify that the log file transmission algorithm can effectively eliminate the redundancy in the log file and reduce the amount of data transmission by 90%.