Multi-Agent Reinforcement Learning for Strategic Bidding in Smart Markets

In a deregulated electricity market, optimal bidding strategies are desired by market participants in order to maximize their individual profits, the optimal bidding strategy for a market participant is difficult to be determined by calculus based methods because of uncertainties and dynamic of electricity market. Power suppliers aim to satisfy two objectives: the maximization of their profit and their utilization rate. To meet with success their goals, they need to acquire a complex behavior by learning through a continuous exploiting and exploring process. Reinforcement learning theory provides a formal framework, along with a family of learning methods. In this project agent-based simulation is employed to study the power market operation under uniform price and discriminatory (pay-as-bid) market. Power suppliers are modeled as adaptive agents capable of learning through the interaction with their environment, following a Reinforcement Learning algorithm. The SA-Q-learning algorithm, a slightly changed version of the popular Q-Learning, is used in this project; it proposes a solution to the difficult problem of the balance between exploration and exploitation and it has been chosen for its quick convergence. Reinforcement learning theory provides a formal framework, along with a family of learning methods. By new state-action definition in a five bus power system and considering SFE model for each player, the player’s strategies in different cases examined.

کلیدواژه ها:

electricity market ، optimal bidding strategies ، Reinforcement learning ، SA-Q-learning algorithm