An in-depth analysis of the impact of processor affinity on network performance

Previous works have shown that in general, performance can be improved by careful affinity of processes/threads to processors in a SMP system. We present a full experimental-based analysis of TCP performance under various affinity modes on SMP servers. Specifically, we made use of mechanisms and interfaces provided by the Redhat Linux-2.4.20 distribution. Best case results (from ttcp bulk-data transfers) showed that interrupt affinity alone provided a throughput gain of up to 25%, and full process and interrupt affinity can achieve gains of 29%. To understand the causes behind the gains, we have broken down the TCP stack into its fundamental logical blocks. This unique view allowed us to showcase exactly where and how affinity affects caching behavior and other architectural events. Where pertinent, we also point out places where affinity has no impact and provide explanations of such.