Estimation of the number of sources based on hypothesis testing

Accurate and efficient estimation of the number of sources is critical for providing the parameter of targets in problems of array signal processing and blind source separation among other such problems. When conventional estimators work in unfavorable scenarios, e.g., at low signal-to-noise ratio (SNR), with a small number of snapshots, or for sources with a different strength, it is challenging to maintain good performance. In this paper, the detection limit of the minimum description length (MDL) estimator and the signal strength required for reliable detection are first discussed. Though a comparison, we analyze the reason that performances of classical estimators deteriorate completely in unfavorable scenarios. After discussing the limiting distribution of eigenvalues of the sample covariance matrix, we proposea new approach for estimating the number of sources which is based on a sequential hypothesis test. The new estimator performs better in unfavorable scenarios and is consistent in the traditional asymptotic sense. Finally, numerical evaluations indicate that the proposed estimator performs well when compared with other traditional estimators at low SNR and in the finite sample size case, especially when weak signals are superimposed on the strong signals.