Lightweight Packing of Log Files for Improved Compression in Mobile Tactical Networks

Devices in mobile tactical edge networks are often resource constrained due to their lightweight and mobile nature, and often have limited access to bandwidth. In order to maintain situational awareness in the cyber domain, security logs from these devices must be transmitted to command and control sites. We present a lightweight packing step that takes advantage of the restricted semantics and regular format of certain kinds of log files to render them substantially more amenable to compression with standard algorithms (especially Lempel-Ziv variants). We demonstrate that we can reduce compressed file sizes to as little as 21% of that of the maximally compressed file without packing. We can also reduce overall compression times up to 64% in our data sets. Our packing step permits lossless transmission of larger log files across the same network transmission medium, as well as permitting existing sets of logs to be transmitted within smaller network availability windows.