Bounds for finite block-length codes

A tight bound for finite block-length codes is required when developing bandwidth efficient systems, and the paper describes an information-theoretic approach for determining the bound for binary codes. It is derived on the assumption that when the entropy loss in a binary symmetric channel equates to that in a soft-decision channel, the corresponding hard and soft-decision decoders must generate the same error probability. The bound agrees well with the Gallager (1968) bound at large block lengths, and is compared to the performance of iteratively decoded product codes. The paper also describes a recursive breakdown of Shannon´s sphere packing bound that permits exact numerical evaluation for information block lengths up to 10³ bits. The difference between the exact solution and Shannon´s approximation is given up to this block length