PSK versus QAM for iterative decoding of bit-interleaved coded modulation

Bit-interleaved coded modulation with iterative decoding (BICM-ID) is a transmission scheme with excellent performance on fading channels. The mapping of the bit patterns to the elements of the signal constellation set (SCS) is the key design element of BICM-ID, especially regarding the asymptotic performance of the BER. We demonstrate that SCSes of PSK (phase shift keying) with optimum bit pattern assignment are significantly superior in the asymptotic error-floor region to the respective square QAM (quadrature amplitude modulation) SCSes. When modifying the PSK SCSes to non-regular SCSes, performance gains of more than 1 dB are realizable in the case of 4 bits-per-channel symbols. The achievable gains are theoretically determined and analyzed, and verified by simulations.