On the pairwise error probability bounds of STTC over Nakagami-m fading channels

We investigate the pairwise error probability (PEP) bounds for space-time trellis codes (STTC) in Nakagami-m fading channels with the arbitrary fading severity parameter, m. We derive an analytical expression for PEP upper bound in the case of two transmit antennas in terms of the higher transcendental functions, such as the Appell hypergeometric function or the Lauricella function, over uncorrelated and correlated Nakagami-m fading channels. Furthermore, we show that the probability bound for the general multiple transmit antenna scenario can also be expressed as an integral involving nested summations.