An improved energy management strategy of fuel cell hybrid vehicles based on proximal policy optimization algorithm

In this research, a novel deep reinforcement learning (DRL) algorithm, i.e. the proximal policy optimization (PPO) is applied to the energy management strategy (EMS) of a fuel cell hybrid vehicle (FCHV) with the dynamic programming (DP) knowledge and parallel computing. The proposed EMS is compared to other EMSs based on typical DRL algorithms including the deep Q-network (DQN) and twin delayed deep deterministic policy gradient (TD3) in terms of the algorithm convergence, FCHV fuel economy, and fuel cell durability. As a typical global optimization method, the DP-based EMS is also introduced as the fuel economy comparison benchmark. Furthermore, the proposed EMS is implemented on the dSPACE platform to verify the deployment feasibility on embedded devices. Results indicate that the hydrogen consumption difference of the PPO-based, DQN-based, and TD3-based EMSs to the DP-based EMS is 3.79 %, 8.45 %, 6.86 % and 1.47 %, 6.34 %, 5.73 % under the training and validation conditions respectively. Besides, compared to the DQN-based and TD3-based EMSs, the PPO-based EMS reduces the fuel cell degradation by 2.51 %, 0.12 % and 5.22 %, 3.27 % under the training and validation conditions respectively. Additionally, the convergence speed of the PPO-based EMS is faster than that of the DQN-based and TD3-based EMSs by 93.55 % and 97.17 %, respectively.