Extraction of crisp logical rules using constructive constrained backpropagation networks

The problem of extraction of crisp logical rules from neural networks trained with a backpropagation algorithm is solved by smooth transformation of these networks into simpler networks performing logical functions. Two constraints are included in the cost function: a regularization term inducing weight decay, and an additional term forcing the remaining weights to ±1 integer values. Networks with minimal number of connections are created, leading to a small number of crisp logical rules. A constructive algorithm is proposed, in which rules are generated consecutively by adding more nodes to the network. Rules that are most general, covering many training examples, are created first, followed by more specific rules, covering a few cases only. This constructive algorithm applied to the iris classification problem generates two rules with three antecedents giving 98.7% accuracy. A single rule for the mushroom problem leads to 98.52% accuracy while three additional rules allow for perfect classification. The rules found for the three monk problems classify all examples correctly