Towards adaptive deep reinforcement learning energy management for electric vehicles: An online updating approach

Deep reinforcement learning is a potential method for online energy management of electric vehicles. However, deep reinforcement learning can suffer from limited training data and underfitting/overfitting characteristics, leading to a compromised vehicle economy in online applications. This paper proposes an online updating approach for deep reinforcement learning-based energy management in electric vehicles with battery/supercapacitor hybrid energy management systems. Firstly, Jensen–Shannon Divergence is introduced to compare the pre-training and online velocity transfer probability surfaces quantitatively and to trigger the online strategy updating when exceeding the threshold. Secondly, existing online driving data is optimized through dynamic programming to construct the optimal state/action dataset, which will be used for actor-network updating. Thirdly, a soft updating method is further proposed to ensure smooth strategy updating for satisfactory generalization ability based on the weighted fusion of multiple networks. Results under Dallas real-world driving cycles validate that the proposed online updating method can reduce the battery degradation and vehicle operation cost by 5.3-10.2% and 4.5-9.0%, respectively, compared with the strategy before updating. This study provides a meaningful attempt at the online updating of deep reinforcement learning-based energy management strategies, which can be easily extended to different vehicle types and driving conditions.