Re-evaluating the performance of the nonlinear prediction error for the detection of deterministic dynamics

The nonlinear prediction error in combination with appropriately generated surrogate data is widely used as a discriminating statistic for weak nonlinearity. In this paper two of the most widely used formulations of this particular measure are evaluated using data from exemplary nonlinear and linear systems. It is found that both methods, contrary to expectation and accepted utility, give rise to substantial false positive and false negative detection rates in purely linear and nonlinear systems respectively. These findings suggest that the nonlinear prediction error should be used with considerable caution. The implications of these findings for the search for nonlinearity in the human electroencephalogram are briefly discussed.