Intelligent energy management for hybrid electric vehicles based on data-driven real-time dynamic programming

Developing effective energy management strategies (EMS) for hybrid electric vehicles (HEVs) in unknown, stochastic driving environments remains a significant challenge. This paper proposes a novel online EMS within a model-based reinforcement learning (MBRL) framework. Central to this framework is a time-variant adaptive virtual environment, integrating an online auto-regressive (AR) predictor for power demand forecasting and a polynomial response surface (PRS) model for powertrain characterization. Synchronously updated via recursive least squares (RLS), these sub-models track the evolving environmental dynamics, thereby eliminating physical trial-and-error risks and providing a continuous interaction platform. Operating within this established virtual environment, a real-time dynamic programming (RTDP) algorithm is employed to execute online policy optimization. Unlike traditional RL methods that attempt to derive global policies across entire driving cycles, the proposed RTDP concentrates computational resources exclusively on learning optimal actions for instantaneous HEV states through direct interactions with this adaptive model. Furthermore, a rule-based guidance mechanism is integrated to mitigate inefficient exploration and accelerate convergence. Validated against dynamic programming (DP), model predictive control (MPC), and the TD3 algorithm under four unknown driving cycles, the strategy achieves near-optimal fuel economy. It limits the performance gap to merely 3.97% – 4.91% relative to the theoretical DP benchmark, significantly outperforming TD3 (6.42%) and MPC (8.40%). Finally, hardware-in-the-loop (HIL) experiments rigorously validate the strategy's practical engineering applicability.