On Convexity of Error Rates in Digital Communications

Convexity properties of error rates of a class of decoders, including the maximum-likelihood/min-distance one as a special case, are studied for arbitrary constellations, bit mapping, and coding. Earlier results obtained for the additive white Gaussian noise channel are extended to a wide class of noise densities, including unimodal and spherically invariant noise. Under these broad conditions, symbol and bit error rates are shown to be convex functions of the signal-to-noise ratio (SNR) in the high-SNR regime with an explicitly determined threshold, which depends only on the constellation dimensionality and minimum distance, thus enabling an application of the powerful tools of convex optimization to such digital communication systems in a rigorous way. It is the decreasing nature of the noise power density around the decision region boundaries that ensures the convexity of symbol error rates in the general case. The known high/low-SNR bounds of the convexity/concavity regions are tightened and no further improvement is shown to be possible in general. The high-SNR bound fits closely into the channel coding theorem: all codes, including capacity-achieving ones, whose decision regions include the hardened noise spheres (from the noise sphere hardening argument in the channel coding theorem), satisfy this high-SNR requirement and thus has convex error rates in both SNR and noise power. We conjecture that all capacity-achieving codes have convex error rates. Convexity properties in signal amplitude and noise power are also investigated. Some applications of the results are discussed. In particular, it is shown that fading is convexity-preserving and is never good in low dimensions under spherically invariant noise, which may also include any linear diversity combining.