Federated deep reinforcement learning for varying-scale multi-energy microgrids energy management considering comprehensive security

The penetration of abundant renewable resources and the integration of distributed energy coupling equipment prompt the development of multi-energy microgrids (MEMGs). However, the uncertainties from the generation and demand sides render the energy management problem of MEMGs a non-trivial task. Centralized and decentralized methods require massive local records containing user information or an accurate system model to derive an optimal solution, which is hard to obtain in practice and the transmission of local data may impose privacy concerns and extra communication burdens. To solve these, a novel physical perception federated learning algorithm is proposed to optimize the scheduling process in MEMGs with different load levels. Considering the model leakage risk during the model transmission procedure, a data encryption process is further introduced to prevent malicious attackers from inferring valid information from the model parameters. A Lagrangian-based deep reinforcement learning method is proposed to ensure safe operation under physical constraints. Therefore, a comprehensive secure energy management paradigm, which considers the security of decision-making, privacy, and data transmission, is constructed in our paper. Extensive experiments based on real-world datasets are conducted to illustrate the effectiveness of our proposed algorithm in terms of cost efficiency, safety, and carbon reduction.