Building the knowledge base of a buyer agent using reinforcement learning techniques

Electronic markets are places where entities not known in advance can negotiate and agree upon the exchange of products. Intelligent agents can be proved very advantageous when representing entities in markets. Mostly, such entities are based on reputation models in order to conclude a transaction. However, reputation is not the only parameter that they could be based on. In this work, we deal with the problem of how and on which entity a buyer should be rely upon in order to conclude a transaction. Reinforcement learning techniques are used for these purposes. More specifically, the Q-learning algorithm is used for the calculation of the reward that the buyer will take for every action in the market environment. Actions represent the selection of specific entities for the negotiation of products. The most important is that the reward values are calculated based on a number of parameters such as the price, the delivery time, etc. The result is a more efficient model that is not based only on the reputation of each entity. Finally, we extend the Q-learning algorithm and propose a methodology for the dynamic Q-table creation which results reduced time for its construction and respectively limited time for the purchase action. Simulations show that this model indicates a significant time reduction in the purchase process in conjunction with the best solution according to the characteristics of products.