Estimation of the directions of arrival of signals in unknown correlated noise. I. The MAP approach and its implementation

The authors propose a method of direction of arrival (DOA) estimation of signals in the presence of noise whose covariance matrix is unknown and arbitrary, other than being positive definite. They examine the projection of the data onto the noise subspace. The conditional probability density function (PDF) of the projected data given the signal parameters and the unknown projected noise covariance matrix is first formed. The a posteriori PDF of the signal parameters alone is then obtained by assigning a noninformative a priori PDF to the unknown noise covariance matrix and integrating out this quantity. A simple criterion for the maximum a posteriori (MAP) estimate of the DOAs of the signals is established. Some properties of this criterion are discussed, and an efficient numerical algorithm for the implementation of this criterion is developed. The advantage of this method is that the noise covariance matrix does not have to be known, nor must it be estimated