Collaborative extremum seeking for welfare optimization

This paper addresses a distributed, model-free optimization problem in the context of multi-agent systems. The set-up comprises of a fixed number of agents, each of which can pick an action and receive/measure a private utility function that can depend on the collective actions taken by all agents. The exact functional form (or model) of the agent utility functions is unknown, and an agent can only measure the numeric value of its utility. The objective of the multi-agent system is to optimize the welfare function (i.e. sum of the individual utility functions). A model-free, distributed, on-line learning algorithm is developed that achieves this objective. The proposed solution requires information exchange between the agents over an undirected, connected communication graph, and is based on ideas from extremum seeking control. A result on local convergence of the proposed algorithm to an arbitrarily small neighborhood of a local minimizer of the welfare function is proved. Application of the solution to distributed control of wind turbines for maximizing wind farm-level power capture is explored via numerical simulations. Also included is a novel analysis of a dynamic average consensus algorithm that may be of independent interest.