Effect of spatial smoothing on the performance of noise subspace methods

The effect of using a spatially smoothed forward-backward covariance matrix on the performance of noise subspace-based methods like MUSIC and the minimum-norm method for the direction-of-arrival (DOA) estimation problem is analyzed. In particular, asymptotic results for the mean-squared error in the estimates of the signal zeros and the DOA are derived. An important outcome of this analysis is that for MUSIC the error in the signal zeros is shown to exhibit a different trend compared to the error in the DOA estimates, leading to difficulty in interpreting the spatial spectrum. For instance, when smoothing is used, the peaks in the spatial spectrum become sharper, giving the impression of higher resolution, whereas in reality the estimates of the DOA may, in fact, have deteriorated compared with the ones obtained using minimal or no smoothing. With regard to the minimum-norm method, the errors in the signal zeros exhibit the same trend as the DOA estimates causing no such problem. The relative comparison of the methods shows that proper spatial smoothing enables the performance of the minimum-norm method to be made comparable to MUSIC