Supervised Neural Network Training with a Hybrid Global Optimization Technique

A novel hybrid global optimization method applied for feedforward neural networks (NN) supervised learning is investigated. The network weights are determined by minimizing the traditional mean-square error function. The optimization technique, called GLP_tauS is a combination of novel global optimization heuristic search based on low-discrepancy sequences of points, called LP_tau Optimization (LP_tauO), a Genetic Algorithm, and a Simplex local search. The proposed method is initially tested on 10 multimodal mathematical functions of 30 and 100 dimensions. Subsequently, it is applied for training moderate size NN for function fitting and solving benchmark classification problems, such as the parity problem (XOR and 4-Parity), Iris dataset, and a medical diagnosis problem (Diabetes). The investigated technique is also tested on predicting continuous output of a mechanical system dataset (Servo). Finally, the results are analysed, discussed, and compared with others.