Multi-market agency-based power procurement strategies for power grid companies using reinforcement learning

At this critical stage of simultaneously advancing the construction of a new power system and power market reform, power grid companies face challenges in managing multifaceted risks and optimizing costs associated with high penetration of renewable energy in agency power procurement. Existing research has mainly studied distributed entities or single markets, which struggles to handle the complex challenges of dynamically linked multidimensional risks and multi-timescale decision-making in agency procurement. Therefore, a collaborative decision-making framework that integrates risk volatility modeling with multi-agent reinforcement learning is proposed. First, a multi-dimensional risk quantification model is developed that incorporates public opinion, supply security and compliance risks. Monte Carlo simulation is subsequently employed to characterize the dynamic distributions of these risks. Second, through risk-cost linkage analysis, an optimization model for agency power purchase transactions is constructed using the Asynchronous Advantage Actor-Critic (A3C) reinforcement learning algorithm, aiming to minimize both procurement costs and associated risks. Finally, case simulations verify the model's effectiveness and explore optimized procurement strategies under different risk scenarios. Simulation results demonstrate that the proposed model reduces procurement costs and total risk by 8.0 %–18.8 %, while reducing the dispersion of risk costs by 30.1 % under a combined risk strategy. This framework offers a methodological tool for enhancing risk control and cost optimization in agency power procurement operations for grid companies.