Carbon-Aware Reinforcement Learning for Cost-Optimal Energy Management of Hydrogen-Electric Heavy-Duty Trucks

The decarbonization of heavy-duty road transportation is a critical component of global carbon neutrality strategies, particularly under the increasing adoption of carbon credit pricing mechanisms such as emissions trading systems. Hydrogen-electric hybrid heavy-duty trucks (H2-E HDTs) offer a promising pathway toward low-carbon freight transport; however, their operational cost-effectiveness strongly depends on coordinated energy management decisions that jointly account for energy prices and carbon emission costs. In this paper, we propose a Carbon-aware Reinforcement Learning based Cost-optimal Energy Management System (CRL-CEMS) for hydrogen-electric heavy-duty trucks operating under carbon pricing environments. The proposed framework explicitly integrates dynamic carbon credit prices into the reinforcement learning decision process, enabling adaptive optimization of hydrogen and electricity usage with respect to both energy economics and carbon economics. A real-time carbon emission estimation model is developed to map hydrogen and electricity consumption into monetary carbon costs based on upstream carbon intensities. The energy management problem is formulated as a continuous control task and solved using the Soft Actor-Critic algorithm, where the agent determines the optimal power split between the fuel cell and battery systems. Extensive simulation experiments are conducted under multiple driving cycles and carbon price scenarios. Simulation results show that CRL-CEMS reduces the total operating cost by 7.2% and lowers carbon emissions by 12.3% compared with a carbon-unaware reinforcement learning baseline, while maintaining stable battery operation. These findings demonstrate that carbon-aware reinforcement learning effectively balances economic and environmental objectives, providing a practical approach for low-carbon, cost-optimal hydrogen-electric heavy-duty truck operation and supporting intelligent fleet energy management under dynamic carbon pricing scenarios.