A modified likelihood function approach to DOA estimation in the presence of unknown spatially correlated Gaussian noise using a uniform linear array

The problem of modified ML estimation of DOAs of multiple source signals incident on a uniform linear array (ULA) in the presence of unknown spatially correlated Gaussian noise is addressed here. Unlike previous work, the proposed method does not impose any structural constraints or parameterization of the signal and noise covariances. It is shown that the characterization suggested here provides a very convenient framework for obtaining an intuitively appealing estimate of the unknown noise covariance matrix via a suitable projection of the observed covariance matrix onto a subspace that is orthogonal complement of the so-called signal subspace. This leads to a formulation of an expression for a so-called modified likelihood function, which can be maximized to obtain the unknown DOAs. For the case of an arbitrary array geometry, this function has explicit dependence on the unknown noise covariance matrix. This explicit dependence can be avoided for the special case of a uniform linear array by using a simple polynomial characterization of the latter. A simple approximate version of this function is then developed that can be maximized via the-well-known IQML algorithm or its variants. An exact estimate based on the maximization of the modified likelihood function is obtained by using nonlinear optimization techniques where the approximate estimates are used for initialization. The proposed estimator is shown to outperform the MAP estimator of Reilly et al. (1992). Extensive simulations have been carried out to show the validity of the proposed algorithm and to compare it with some previous solutions