Mutual information neuro-evolutionary system (MINES)

This article presents a new approach for automatically determining the optimal quantity and connectivity of the hidden-layer of a three-layer Feed-Forward Neural Network (FFNN) based on a theoretical and practical approach. The system (MINES) is a combination of Neural Network (NN), Back-Propagation (BP), Genetic Algorithm (GA), Mutual Information (MI), and clustering. BP is used to reduce the training-error while MI aides BP to follow an effective path. A GA changes the incoming synaptic connections of the hidden-nodes based on MI fitness. Assigning MI as the fitness of individuals brings a competition between hidden-nodes to acquire a higher amount of information from the error-space. Weight clustering is applied to reduce those hidden-nodes having similar weights. Experimental results are presented, and future directions discussed.