Estimation of a generalized non-parametric magnitude squared coherence spectrum using the GLRT-based rank detection

Several seemingly disparate non-parametric magnitude squared coherence (MSC) estimators are treated in a unified way recently. This paper gives a new insight into the non-parametric MSC estimators and points out that the reduced-rank techniques can be applied to the coherence matrix, where the rank is determined by the generalized likelihood ratio test (GLRT)-based rank detection method. Numerical simulation results verify that the non-parametric MSC estimators combined with the GLRT-based rank detection method could effectively suppress the non-correlated signals and improve the estimation accuracy.