Nonparametric trellis equalization in the presence of non-Gaussian interference

We consider the problem of trellis equalization of the intersymbol interference channel in the presence of thermal noise and cochannel interference (CCI). Conventional maximum-likelihood sequence estimation (MLSE) and maximum a posteriori probability (MAP) trellis equalizers treat the sum of noise and interference as additive white Gaussian noise, while CCI is generally a colored non-Gaussian process. We propose a novel nonparametric approach based on the estimation of the probability density function of the noise-plus-interference. Given the availability of a limited volume of data, the density is estimated by kernel-smoothing techniques. The use of a whitening filter in the presence of temporally colored disturbance is also addressed. Simulation results are provided for the global system for mobile communications (GSM), showing a significant performance improvement with respect to the equalizer based on the Gaussian assumption. Major advantages of the proposed strategy are its intrinsic robustness and general applicability to those cases where accurate modeling of the interference is difficult or a model is not available.