Deep reinforcement learning based adaptive energy management for plug-in hybrid electric vehicle with double deep Q-network

The equivalent consumption minimization strategy (ECMS) with pre-calibrated constant equivalence factor (EF) can ensure near global optimal solution for certain driving cycle and enable good real-time capability, but it is difficult to adapt to a wide range of driving conditions. To this end, aiming at the optimal energy management problem of a plug-in hybrid electric vehicle (PHEV), this paper proposes a deep reinforcement learning (DRL) based adaptive ECMS by combing the double deep Q-network (DDQN) and the driving cycle information. The DDQN is applied to correct the EF of the ECMS in a feed-forward manner with the battery state-of-charge (SOC) and the periodic predicted driving cycle information as inputs, and the ECMS is utilized to calculate the engine torque and gear ratio of the powertrain. The driving cycle information is represented by the average velocity, which is predicted by the historical velocity sequence based on the back-propagation (BP) neural network, and the difference of the average velocity between two continuous time windows. The hardware-in-the-loop (HIL) platform is constructed to test the performance of the proposed strategy. It is shown that the future average velocity can be well predicted by the historic velocity sequence. Both simulation and HIL test results demonstrate that the proposed adaptive ECMS based on DDQN exhibits superior performance in improving the vehicle fuel economy.