Bayesian evolution of rich neural networks

In this paper we present a genetic approach that uses a Bayesian fitness function to the design of rich neural network topologies in order to find an optimal domain-specific non-linear function approximator with good generalization performance. Rich neural networks have a feed-forward topology with shortcut connections and arbitrary activation functions at each layer. This kind of topologies is particularly well suited for non-linear regression tasks, but it may suffer for overfilling issues. In this paper we present a Bayesian fitness function to effectively apply genetic algorithms with these models obtaining, in a completely automated way, models well-matched to the problem, with good generalization capability, and low complexity.