Deep reinforcement learning based coordinated control for integrated energy system with photovoltaic, storage and electric vehicles considering transportation-power network couplings

In coupled transportation–power networks, the intermittency of renewable generation and the charging demand uncertainties of Electric Vehicles (EVs) present major challenges, highlighting the importance of coordinated operation among multiple flexible and dispatchable resources. This paper developed a deep reinforcement learning based framework to coordinate the operation of photovoltaic (PV), energy storage units (ESUs) and EVs, considering the coupling interactions between the transportation and power networks. For PV inverters, a stability-constrained soft actor critic method grounded in the LaSalle invariance principle is extended for voltage control. For EVs, a two-stage scheduling scheme is proposed: a piecewise-linear user equilibrium traffic assignment model is formulated, after which a short-term revenue-oriented EV scheduling model is further developed as a mixed-integer linear program. For ESUs, a SoC-guided hierarchical control mechanism is established, with the upper layer forecasting target SoC trajectories via a hybrid model, and the lower layer embedding them into the reward function to guide ESUs. The proposed solution is validated on the Nguyen transportation network coupled with the IEEE 33-bus and the 141-bus network, respectively. The numerical results demonstrate the effectiveness of the proposed solution in maintaining grid stability and ensuring economic benefits for users.