Neural nonlinear classifiers with synaptic weight commitment

We develop a theory for immediately detecting patterns violating linear separability in a training set, as soon as they are presented to the classifier. The theory is based on the computation of the solution cone in the weight space of a single linear threshold unit (LTU). The separability-violating patterns can be skipped and in the end, we obtain a linearly separable subset of the original training set together with the subset´s solution cone. We further propose an iterative algorithm for computing the solution cone and we introduce the concept of garbage collection for removing redundancy from the cone representation. The algorithm can be implemented by a novel neural network architecture where the synaptic weights have constant values throughout the life of the synapse. We show that by combining multiple such models we can construct a larger network capable of learning the classification boundaries of convex classes, therefore solving an important class of nonlinearly separable problems