Application of SPSA techniques in nonlinear system identification

Simultaneous perturbation stochastic approximation (SPSA) is an optimization method which requires only a few objective function evaluations to obtain gradient information. In this paper, a first-order SPSA algorithm is described, which makes use of several numerical artifices, including adaptive gain sequences, gradient smoothing and a step rejection procedure, to enhance convergence and stability. This algorithm is particularly well suited to problems involving a large number of parameters and its potentialities are demonstrated in the context of nonlinear system identification. First, a relatively simple example is considered, i.e. the development of a neural network state space model for a level-control system. Second, a more advanced application is studied, i.e. the estimation of the most-likely kinetic parameters and initial conditions of a bioprocess model describing the evolution of a few macroscopic components in batch animal cell cultures.