Evolutionary hybrid composition of activation functions in feedforward neural networks

Considering computational algorithms available in the literature, associated with supervised learning in feedforward neural networks, a wide range of distinct approaches can be identified While the adjustment of the connection weights represents an omnipresent stage, the algorithms differ in three basic aspects: the technique chosen to determine the dimension of the multilayer neural network, the procedure adopted to determine the activation function of each neuron, and the kind of composition of the hidden activations used to produce the output. The advanced learning algorithms are designed to treat all these three aspects during learning, guiding to dedicated solutions. In this paper, an evolutionary hybrid learning algorithm is presented to deal simultaneously with these three aspects. The essence of this approach is the existence of a search procedure based on a synergy between genetic algorithms and conjugate gradient optimization