A system-level optimization framework for high-performance networking

Processing data in a streaming fashion is involved in many applications, including radar, electro-optical and infrared imaging, and other scenarios in which real-time data is acquired from sensors. While the difficulty of processing such data in real-time or in an otherwise timely fashion is the topic of much research, transferring the data between machines or devices is also a key challenge. This work investigates the utilization of commodity high-speed networking products - in particular, 40 gigabit Ethernet - for supporting high-speed data transfer in streaming applications. For such systems, optimizing the system configuration to support data transfers approaching line rate is critical. This work demonstrates the use of an optimization framework to explore the impact of various system-level optimization settings, including kernel module settings, network interface driver-level settings, kernel buffer sizes, thread and CPU core scheduling, non-uniform memory access (NUMA) scheduling, interrupt handling, and CPU power-saving state management.