Normalized natural gradient adaptive filtering for sparse and non-sparse systems

This paper introduces a class of normalized natural gradient algorithms (NNGs) for adaptive filtering tasks. Natural gradient techniques are useful for generating relatively simple adaptive filtering algorithms where the space of the adaptive coefficients is curved or warped with respect to Euclidean space. The advantage of normalizing gradient adaptive filters is that constant rates of convergence for signals with wide dynamic ranges may be achieved. We show that the so-called proportionate normalized least mean squares (PNLMS) algorithm, an adaptive filter that converges quickly for sparse solutions, is in fact an NNG on a certain parameter space warping. We also show that by choosing a warping that favors diverse or dense impulse responses, we may obtain a new adaptive algorithm, the inverse proportionate NLMS (INLMS) algorithm. This procedure converges quickly to and accurately tracks non-sparse impulse responses.