Lean multi-agent deep reinforcement learning for uncertainty handling in the energy management of networked microgrids

This study introduces a Lean Multi-Agent Deep Reinforcement Learning (L-MADRL) framework for energy management of networked microgrids (NMGs) with multiple electricity retailers (ERs) and microgrids (MGs) under renewable and load uncertainties. Coordinating energy exchanges in such systems is challenging due to the need for market efficiency, technical feasibility, and scalability. The proposed framework combines a multi-agent Deep Q-Network (DQN) with a single-level reformulation of a bi-level optimization model. In this formulation, the upper level maximizes ER profits and the network’s available transfer capability (ATC), while the lower-level MG cost minimization is replaced by Karush–Kuhn–Tucker (KKT) conditions, yielding a mathematical program with equilibrium constraints (MPEC). This hybrid design offers two benefits: (i) technical constraints such as power flow limits, generator capacities, and market rules are embedded in the MPEC, freeing DRL agents from constraint enforcement and improving learning stability and policy reliability, and (ii) explicit ATC consideration enhances power transfer efficiency and enables network-aware coordination. Performance was evaluated on PJM 5-bus and IEEE 14-bus test systems against deterministic, risk-neutral (RNSO), and risk-averse (RASO) stochastic optimization. Results show that in the 5-bus case, L-MADRL reduced MG costs by 10.3% and increased ER profits by 3.7%, while in the 14-bus case costs decreased by 2.6% and profits rose by 11.4%. L-MADRL also improved ATC, exceeding the best benchmark by 32% in the 5-bus system and by 30% initially and 10% at peak in the 14-bus system. Across all cases, runtimes remained below 3 s, highlighting the framework’s scalability and computational efficiency.