Characterization and computation of optimal distributions for channel coding

This paper concerns the structure of capacity-achieving input distributions for stochastic channel models, and a renewed look at their computational aspects. The following conclusions are obtained under general assumptions on the channel statistics. i) The capacity-achieving input distribution is binary for low signal-to-noise ratio (SNR). The proof is obtained on comparing the optimization equations that determine channel capacity with a linear program over the space of probability measures. ii) Simple discrete approximations can nearly reach capacity even in cases where the optimal distribution is known to be absolutely continuous with respect to Lebesgue measure. iii) A new class of algorithms is introduced based on the cutting-plane method to iteratively construct discrete distributions, along with upper and lower bounds on channel capacity. It is shown that the bounds converge to the true channel capacity, and that the distributions converge weakly to a capacity-achieving distribution.