Safe off-policy reinforcement learning on eco-driving for a P2-P3 hybrid electric truck

With the rapid development of intelligent transportation system(ITS) and network technology, vehicles can receive more information about traffic conditions. Although many researchers have conducted the research on eco-driving, many methods ignore the unsafety of reinforcement learning agent or use complex mechanism to improve behavior or they set simple reward function to express the relationship between the SOC and fuel consumption. This paper proposes a safe-off policy eco-driving strategy(SOPEDS) based on a hierarchical frame for a hybrid electric truck in a truck-following scenario in order to solve the problems. Firstly, the truck-following strategy in the upper layer is designed based on the deep deterministic policy gradient(DDPG) algorithm. Then the relevant safe mechanism is introduced in order to reduce the collision risk. In the lower layer, the energy management strategy designed by using reinforcement learning(RL) algorithm. And a new function is designed to shape the reward function to guide agent learning. Finally, the performance of the proposed method is proved by experiments. The results show the fuel saving rate can reach 97.46% of the DP. Our research improved driving safety and introduced a reward function design method which is meaningful for RL training processing for other vehicles.