Learning the IPA Market with Individual and Social Rewards

Market-based mechanisms offer a promising approach for distributed allocation of resources without centralized control. One of those mechanisms is the iterative price adjustment (IPA). Under standard assumptions, the IPA uses demand functions that do not allow the agents to have preferences over some attributes of the allocation, e.g. different price or resource levels. One of the alternatives to address this limitation is to describe the agents´ preferences using utility functions. In such a scenario, however, there is no unique mapping between the utility functions and a demand function. Gomes & Kowalczyk [10, 9] proposed the use of Reinforcement Learning to let the agents learn the demand functions given the utility functions. Their approach is based on the individual utilities of the agents at the end of the allocation. In this paper, we extend such a work by applying a new reward function, based on the social welfare of the allocation, and by considering more clients in the market. The learning process and the behavior of the agents using both reward functions are investigated through experiments and the results compared.