Are ensemble-average learning curves reliable in evaluating the performance of adaptive filters?

We treat the computation of the learning curves of the LMS algorithm by simulation (that is, the computation of the MSE as a function of the time instant). Since closed-form analytic expressions for learning curves are quite hard to obtain in most practical situations, one usually approximates learning curves by performing several repeated experiments and by averaging the resulting squared-error curves. We show, both by examples and analytically, that when the step-size is large, this approximation of the MSE can be misleading. This is contrary to what one would expect, given the excellent agreement one obtains between simulations and theory for small step-sizes and independent inputs, even using only as few as 10 experiments. The theoretical analysis explains both the good results obtained for small step-sizes, and the discrepancies that arise for large step-sizes.