Constellation Design for Trellis-Coded Unitary Space–Time Modulation

We consider the problem of creating signal constellations for trellis-coded unitary space–time communication links, where neither the transmitter nor the receiver knows the fading gains of the channel. Our study includes design techniques for trellis-coded schemes with and without parallel paths, which allows us to find a tradeoff between low complexity and high performance. We present a new formulation of the constellation design problem for trellis-coded unitary space–time modulation schemes. The two key differences in our approach against those of other authors are that we not only combine the constellation design and mapping by set partitioning into one step, but we also use directly the Chernoff bound of the pairwise error probability as a design metric. By novelly employing a theorem for the Clarke subdifferential of the sum of the largest singular values of the unitary matrix, we also present a numerical optimization procedure for finding signal constellations resulting in high-performance communications systems. To demonstrate the advantages of our new design method, we report the best constellations found for trellis-coded unitary space–time modulation systems. Simulation results show that these constellations achieve 1 dB coding gain against the usually used constellations.