Parametric design space exploration for optimizing QAM based high-speed communication

The growing demands on high-speed data transmission over long distances require optimized communication systems with bandwidth efficient modulation techniques like Quadrature Amplitude Modulation (QAM). However, the various design parameters of these QAM based systems conflict each other and the complex interdependence makes it difficult and time consuming to appropriately tune the parameters towards an optimal solution for a given set of constraints. Therefore, throughout this paper we analyze various conflicting design parameters of QAM and derive objective optimized trade-offs. More specifically, we study in detail the effects and interdependencies of parameters in the matched filter, the Forward Error Correction (FEC) code pair and the modulation order and span the design and solution space through experimental and numerical results. We discuss the observed effects and show how a selection of parameters can be concluded under various constraints. Furthermore, a pareto optimality search on our design space points out the best combinations of modulation order, filter and FEC design parameters for maximizing the performance under various Signal-to-Noise Ratio (SNR) conditions of the channel. For instance, if the channel is constrained by a SNR of 10dB and the tolerable Bit Error Rate (BER) is 10^-3, then 16-QAM with a required 2/3 FEC yields 33.3% higher performance than 64-QAM with a required 1/3 FEC. Thus, the outcomes of this paper are a comprehensive elaboration on the cross-effects of conflicting design parameters and a set of optimized solutions under various constraints.