Optimal trellis-coded 8-PSK and 4-AM modulations for the Rayleigh channel

Investigates two families of trellis codes having the spectral efficiency of uncoded QPSK (2 bits/s/Hz) for the Rayleigh fading channel assuming ideal channel state information and coherent detection. The first family employs an 8-PSK modulation and the second one is based on independent coding of the I&Q components using the 16-QAM signal set (4-AM for each component). For each family, optimal codes were found using systematic code search which minimizes the asymptotic error event probability rather than maximizing the minimum squared product distance corresponding to the code diversity. Analytical performances of these codes are evaluated in both Gaussian and Rayleigh channels. The results indicate that for similar complexity, 4-AM trellis codes designed for the Rayleigh channel outperform significantly 8-PSK trellis codes in terms of diversity while achieving the rotational invariance property