Classification with linear networks using an online constrained LDA algorithm

Principal component analysis (PCA) and linear discriminant analysis (LDA) are two statistical tools utilized in many signal processing areas such as data compression and pattern recognition. Online algorithms, such as Oja´s rule, have found wide application for PCA and it has been generalized in our previous paper (1997) to LDA (Fisher criterion) using the framework of gradient descent learning. In this paper, the rule is further extended to accept the case of constraints in the feature extraction stage as is often necessary for real world applications. As examples, the new rule is applied to regularizers in the form of both a 2D Gaussian filter for handwritten digit classification and determination of the memory depth of the Gamma filter for isolated word recognition. Results show the good behavior of the learning rule and the advantage of using regularization for improved generalization