Deep reinforcement learning-based multi-objective energy management system for microgrids under flexible energy market

The promotion of renewable energy in Microgrids (MGs) brings uncertainty from weather and climate change, thereby challenging the energy management systems (EMSs) in energy allocation tasks. This paper proposes a reinforcement learning (RL)-based online EMS (RLEMS) approach to solve the multi-objective optimization of power flow in an MG with a high proportion of uncertainty. A novel online energy management framework is proposed in the RLEMS, which utilizes the information extraction ability of RL algorithms to map the multi-dimensional MG states with optimal power flow operations, diluting the time sequence self-correlation in solving uncertainty. Different from conventional time series optimization methods, the proposed RLEMS can be deployed with few-shot historical datasets and operate the MG with a high resolution. The global economic horizon of the RLEMS is established through a novel energy price evaluation model (EPEM) instead of conventional prediction approaches, in which the vague evaluation of electricity price is quantified and projected to the RLEMS timescale and the MG scope. The multi-timescale uncertainty and multi-dimensional MG status are combined with distinguished priorities by the EPEM and RLEMS reward function. The performance of the proposed RLEMS is validated through realistic numerical experiments. The proposed RLEMS can maintain the stable and economical operation of the MG for over 240 hours continuously with only 24-hour training data. The comparison with cutting-edge approaches indicates that the proposed RLEMS provides promising cost-saving performance.