An efficient energy management strategy of a hybrid electric unmanned aerial vehicle considering turboshaft engine speed regulation: A deep reinforcement learning approach

Low-altitude economy, with its great potential, can be applied widely in different areas and promote the development of various industries. As the main carriers in this strategic emerging economy, large-sized unmanned aerial vehicles (UAVs) keep gaining attention due to their high mobility and broad range of applications. To extend flight range, hybrid electric UAVs are emerging as a promising solution, leveraging energy-saving potential through energy management strategy (EMS). Most research on energy-saving hybrid electric systems either overlooks the importance of turboshaft engine speed regulation or incorporates it into the EMS without considering the dwell time constraints (DTCs) of speed changes. DTCs align with engine speed regulation which are crucial for ensuring the safe operation of the turboshaft engine. Without DTCs, such control strategies may lead to suboptimal real-world performance and even engine failure. However, integrating DTCs into the control problem introduces high nonlinearity and computational complexity which is difficult to solve in an optimal control problem. To address such issues, an efficient reinforcement control strategy is proposed to optimize both energy management and turboshaft engine speed regulation for hybrid electric UAVs. First, a mathematical model of the hybrid powertrain with a turboshaft engine generator set is established. Second, a clipped proximal policy optimization agent is developed to solve the optimal EMS control problem considering engine speed regulation. Especially, an action mapping and a minimum DTC programming method are proposed to enhance convergence and maintain system safety. Third, real-time flight cycles from our prototype UAV are incorporated into the training loop to accurately reflect actual power flow during flight. Finally, the effectiveness and efficiency of the proposed control strategy are validated through simulation. Results demonstrate the effectiveness of the proposed algorithm, which is nearly 95 % close to the globally optimal solution. And the real-time control performance of the proposed strategy is verified through hardware-in-the-loop experiments.