Pareto-based Multi-Objective Machine Learning

Machine learning is inherently a multi-objective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multi-objective optimization methodology have gained increasing impetus, particularly thanks to the great success of multi-objective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multi-objective learning approaches are more powerful compared to learning algorithms with a scalar cost functions in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. This talk provides first a brief overview of Pareto-based multi-objective machine learning techniques. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multi-objective ensemble generation are compared and discussed in detail. Most recent results on multi-objective optimization of spiking neural networks will be presented.