Using a Modified Erev-Roth Algorithm in an Agent-Based Electricity Market Model

One of the important tools for the analysis and prediction of operation of electricity markets are agent-based models, which simulate the behavior of decentralized agents (for example, producers and buyers), each with its own objectives and means. In these models learning of agents submitting price bids to a wholesale market plays an important role. In the process of repeated interaction an agent adapts to the environment and to the behavior of other agents, learns to predict the results of its own actions. The paper presents a modification of the classical Erev-Roth reinforcement learning algorithm which takes into account the distance between alternatives. The proposed modified algorithm is used to represent agents' learning in an agent-based model of the Russian wholesale electricity market (Siberian pricing zone) within the bounds of the day-ahead market. It is shown that it has some significant advantages as compared to the original algorithm. In particular, the algorithm is naturally interpretable, is robust to the choice of discretization step, is invariant to a shift in payoffs scale. On the whole, the algorithm is more flexible than the original one. When the modified algorithm is used, one observes good coherence between the dynamics of model price and the observable dynamics of the price in the market.