Maximum-likelihood array processing in non-Gaussian noise with Gaussian mixtures

Many approaches have been studied for the array processing problem when the additive noise is modeled with a Gaussian distribution, but these schemes typically perform poorly when the noise is non-Gaussian and/or impulsive. This paper is concerned with maximum likelihood array processing in non-Gaussian noise. We present the Cramer-Rao bound on the variance of angle-of-arrival estimates for arbitrary additive, independent, identically distributed (iid), symmetric, nonGaussian noise. Then, we focus on non-Gaussian noise modeling with a finite Gaussian mixture distribution, which is capable of representing a broad class of non-Gaussian distributions that include heavy tailed, impulsive cases arising in wireless communications and other applications. Based on the Gaussian mixture model, we develop an expectation-maximization (EM) algorithm for estimating the source locations, the signal waveforms, and the noise distribution parameters. The important problems of detecting the number of sources and obtaining initial parameter estimates for the iterative EM algorithm are discussed in detail. The initialization procedure by itself is an effective algorithm for array processing in impulsive noise. Novel features of the EM algorithm and the associated maximum likelihood formulation include a nonlinear beamformer that separates multiple source signals in non-Gaussian noise and a robust covariance matrix estimate that suppresses impulsive noise while also performing a model-based interpolation to restore the low-rank signal subspace. The EM approach yields improvement over initial robust estimates and is valid for a wide SNR range. The results are also robust to PDF model mismatch and work well with infinite variance cases such as the symmetric stable distributions. Simulations confirm the optimality of the EM estimation procedure in a variety of cases, including a multiuser communications scenario. We also compare with existing array processing algorithms for non-Gaussian noise.