Cochannel interference mitigation using whitening receiver designs in bandlimited microcellular wireless systems

The problem of cochannel interference (CCI) suppression using a noise-plus-interference whitening matched filter (WMF) is addressed. Binary phase-shift keying modulated transmission over a Nakagami-m fading channel is considered in a bandlimited microcellular wireless system. It is shown that the signal-to-interference-plus-noise ratio (SINR) of the WMF can not exceed that of the conventional matched filter (CMF) receiver for synchronous CCI, but in the asynchronous channel, CCI whitening can be used to improve receiver performance. The SINR-maximizing filter (SINRMF) in an asynchronous channel is derived and shown to be composed of a WMF followed by a discrete time filter. The SINRs of the WMF and SINRMF receivers are analytically evaluated for a standard raised-cosine (RC) and a Beaulieu-Tan-Damen pulse. It is shown that for the RC pulse, the WMF can achieve a SINR gain as large as 1.76 dB over the CMF receiver provided that the transmission is free of intersymbol interference (ISI). The SINR of the ISI-impaired system is also evaluated and the conditions under which this system can achieve close to ISI-free SINR are studied. The SINR is evaluated for the SINRMF and it is shown that this filter can restore much of the SINR loss incurred due to ISI introduced by the WMF.