Adaptive infinite-horizon control of hybrid EV/FCEV charging hubs: A large-model based deep reinforcement learning approach

The combination of hydrogen refueling facilities for fuel cell electric vehicles (FCEVs) with dispersed electric vehicle (EV) charging stations provides an economically viable and flexible scheme to decarbonize the modern transportation systems. In this paper, to seek the long-run profitable operation of such EV/FCEV Charging Hubs, we design an adaptive infinite-horizon controller using a large-model based deep reinforcement learning (DRL) method. To realize the infinite time horizon control in response to stochastic vehicles’ arrival and varying energy prices, an identical initial state constrained Markov decision process (I2S-CMDP) problem is formulated. To address the computational challenges imposed by I2S constraints, the Lagrangian relaxation method is applied. By analyzing its structural properties, we show that there exists a set of fixed Lagrangian multipliers that are optimal when some conditions are satisfied. Then, a Gated Attention Actor Critic (GTAC) DRL algorithm is presented. The incorporation of the large model, implemented as a gated transformer, ensures bounded model updates, contributing to enhanced stability. Numerical studies on exemplary EV/FCEV Charging Hubs validate the effectiveness of the proposed method by achieving higher training stability and superior solution performance compared to traditional DRL algorithms. For our two test instances, the variances of rewards during DRL training have been reduced by 84.15 % and 94.84 % respectively, demonstrating the advantages of the GTAC algorithm in significantly higher training stability. Moreover, the hybrid charging system outperforms the standalone one, with an increase in operational profit of 13.6 %.