Efficient Bidding Strategies for Simultaneous Cliff-Edge Environments

This paper proposes an efficient agent for competing in simultaneous substitutional cliff-edge (SCE) environments, which include simultaneous auctions and multi-player Ultimatum-Games. The agent competes in one-shot interactions repeatedly, each time against different human opponents, and its performance is evaluated based on all the interactions in which it participates. It learns the general pattern of the population´s behavior and does not apply any examples of previous interactions in the environment, neither of other competitors nor of its own. Moreover, the agent rapidly adjusts to environments comprising a large number of optional decisions at each decision point. We propose a generic approach which competes in different substitutional environments under the same configuration, with no knowledge about the specific rules of each environment. The underlying mechanism of the proposed agent is the simultaneous deviated virtual reinforcement learning (SDVRL) algorithm, which is an extension of an algorithm for non-simultaneous environments. In addition, we propose a heuristic for improving our agent´s complexity. Experiments comparing the average payoff of the proposed algorithm with other possible algorithms reveal a significant superiority of the former. In addition, our agent performs better than human competitors executing the same tasks.