Factors affecting the reliability of transfer function estimates of cardiovascular variability signals

The reliability of transfer function (TF) estimates has been traditionally judged according to the threshold value of 0.5 in the magnitude squared coherence (MSC). However, other parameters affect the reliability as well, and low MSC values are often unavoidable in physiological systems. The study was devised (1) to assess by computer simulations the applicability of theoretical expressions for the error of TF estimates, (2) to provide error curves for TF estimates as a function of major experimental and analysis parameters and (3) to evaluate the fidelity of observed TF shapes with respect to expected ones when the MSC is low. Theoretical confidence intervals for the TF magnitude were found to be accurate enough to be used in practical analysis. Curves for the normalized bias and SD of TF estimates were derived, providing useful information for appropriate design of data collection and analysis. With an appropriate choice of analysis parameters the shape of observed TFs were found to be close to the expected ones even when the MSC is <0.5