Comparison of tracking algorithms for single layer threshold networks in the presence of random drift

This paper analyzes the behavior of a variety of tracking algorithms for single-layer threshold networks in the presence of random drift. We use a system identification model to model a target network where weights slowly change and a tracking network. Tracking algorithms are divided into conservative and nonconservative algorithms. For a random drift rate of γ, we find upper bounds for the generalization error of conservative algorithms that are 𝒪(γ ^2/3) and for nonconservative algorithms that are 𝒪(γ). Bounds are found for the perceptron tracker and the least mean square (LMS) tracker. Simulations show the validity of these bounds and show that the bounds are tight when γ is small and the number of inputs n is large. These results show that the perceptron tracker and the LMS tracker can work well in slowly changing nonstationary environments