The never ending problem of counting bits efficiently

The problem of counting the ones of a bit sequence (commonly known as the “popcount” function) is one of the most interesting problems in cryptography. Although it has received attention for a very long time, it was brought back into focus by the new developments in hardware, both for CPUs (SSE 4.2) and for GPUs. The problem is of special interest in the field of testing the quality of random sequences and occurs in the first tests of the NIST battery of statistical tests for randomness. Obviously, for a “good” random sequence we would expect a “popcount” of about half the length of that sequence. Since testing the quality of a random number generator is notoriously computationally intensive we are of course interested in implementing efficiently the “popcount” function. In the present paper we analyze several implementation methods in search of the most efficient one; for now, we restrict our attention to the serial implementations for CPUs.