IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v324y2025ics0360544225015749.html
   My bibliography  Save this article

Comprehensive tradeoff and utilization of airborne renewable energy and uncertain stratospheric wind potential based on reinforcement learning

Author

Listed:
  • Liu, Yang
  • Lv, Mingyun
  • Sun, Kangwen

Abstract

Solar-powered airships are demonstrated overwhelming superiority in plentiful application scenario. Adaptation to the flight environment and efficient energy management are essential during the mission. To improve the operating efficiency of airborne energy system, the tradeoff and integration of airborne renewable energy and uncertain stratospheric wind potential is studied. To complete the station keeping mission utilizing external and internal energy which has complex decision support parameters in different scales and continuous control action spaces with different characteristics, a Noisy Heterogeneous Policy Network Proximal Policy Optimization method is proposed. The state standardization, piecewise reward function, output action with noise, and heterogeneous policy network are designed. The results show that the proposed method has better convergence speed under different degrees of uncertainty of wind field and at different starting points. When the prediction error of the wind velocity is less than 2 m/s, the effective time within the region of the airship starting at specific positions is more than 80 %. When the error reaches 5 m/s, the time percentage is reduced to 50 %. The research results of this paper can provide some valuable reference for improving the performance of renewable energy system on stratospheric airship during the long-time flight in uncertain wind fields.

Suggested Citation

  • Liu, Yang & Lv, Mingyun & Sun, Kangwen, 2025. "Comprehensive tradeoff and utilization of airborne renewable energy and uncertain stratospheric wind potential based on reinforcement learning," Energy, Elsevier, vol. 324(C).
    • Handle: RePEc:eee:energy:v:324:y:2025:i:c:s0360544225015749
    DOI: 10.1016/j.energy.2025.135932
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544225015749
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2025.135932?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Liu, Yang & Sun, Kangwen & Lv, Mingyun, 2024. "Mission-oriented dynamic reconfiguration of airborne photovoltaic array based on multidisciplinary model," Renewable Energy, Elsevier, vol. 234(C).
    2. Marc G. Bellemare & Salvatore Candido & Pablo Samuel Castro & Jun Gong & Marlos C. Machado & Subhodeep Moitra & Sameera S. Ponda & Ziyu Wang, 2020. "Autonomous navigation of stratospheric balloons using reinforcement learning," Nature, Nature, vol. 588(7836), pages 77-82, December.
    3. Siyu, Liu & Kangwen, Sun & Jian, Gao & Haoquan, Liang, 2023. "Receiving energy analysis and optimal design of crystalline silicon solar cell array on solar airship," Energy, Elsevier, vol. 282(C).
    4. Lu, Ruyuan & Li, Xin & Chen, Ronghao & Lei, Aimin & Ma, Xiaoming, 2024. "An Alternative Reinforcement Learning (ARL) control strategy for data center air-cooled HVAC systems," Energy, Elsevier, vol. 308(C).
    5. Song, Tiewei & Fu, Lijun & Zhong, Linlin & Fan, Yaxiang & Shang, Qianyi, 2024. "HP3O algorithm-based all electric ship energy management strategy integrating demand-side adjustment," Energy, Elsevier, vol. 295(C).
    6. Liu, Yang & Sun, Kangwen & Xu, Ziyuan & Lv, Mingyun, 2022. "Energy efficiency assessment of photovoltaic array on the stratospheric airship under partial shading conditions," Applied Energy, Elsevier, vol. 325(C).
    7. Jia, Chunchun & He, Hongwen & Zhou, Jiaming & Li, Jianwei & Wei, Zhongbao & Li, Kunang, 2024. "Learning-based model predictive energy management for fuel cell hybrid electric bus with health-aware control," Applied Energy, Elsevier, vol. 355(C).
    8. Jia, Chunchun & Zhou, Jiaming & He, Hongwen & Li, Jianwei & Wei, Zhongbao & Li, Kunang, 2024. "Health-conscious deep reinforcement learning energy management for fuel cell buses integrating environmental and look-ahead road information," Energy, Elsevier, vol. 290(C).
    9. Zhang, M.Y. & Chen, J.J. & Yang, Z.J. & Peng, K. & Zhao, Y.L. & Zhang, X.H., 2021. "Stochastic day-ahead scheduling of irrigation system integrated agricultural microgrid with pumped storage and uncertain wind power," Energy, Elsevier, vol. 237(C).
    10. Rehman, Anis Ur & Ullah, Zia & Qazi, Hasan Saeed & Hasanien, Hany M. & Khalid, Haris M., 2024. "Reinforcement learning-driven proximal policy optimization-based voltage control for PV and WT integrated power system," Renewable Energy, Elsevier, vol. 227(C).
    11. Yang, Xixiang & Liu, Duoneng, 2017. "Renewable power system simulation and endurance analysis for stratospheric airships," Renewable Energy, Elsevier, vol. 113(C), pages 1070-1076.
    12. Huang, Xuejin & Zhang, Jingyi & Ou, Kai & Huang, Yin & Kang, Zehao & Mao, Xuping & Zhou, Yujie & Xuan, Dongji, 2024. "Deep reinforcement learning-based health-conscious energy management for fuel cell hybrid electric vehicles in model predictive control framework," Energy, Elsevier, vol. 304(C).
    13. Zare, Aramchehr & Boroushaki, Mehrdad, 2024. "A knowledge-assisted deep reinforcement learning approach for energy management in hybrid electric vehicles," Energy, Elsevier, vol. 313(C).
    14. Guo, Chenyu & Wang, Xin & Zheng, Yihui & Zhang, Feng, 2022. "Real-time optimal energy management of microgrid with uncertainties based on deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    15. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiang, Yi & Li, Jun, 2025. "Reconfiguration strategy for solar array system on stratospheric airship based on cost-benefit analysis," Renewable Energy, Elsevier, vol. 243(C).
    2. Lan, Penghang & Chen, She & Li, Qihang & Li, Kelin & Wang, Feng & Zhao, Yaoxun, 2024. "Intelligent hydrogen-ammonia combined energy storage system with deep reinforcement learning," Renewable Energy, Elsevier, vol. 237(PB).
    3. Shan, Chuan & Sun, Kangwen & Ji, Xinzhe & Cheng, Dongji, 2023. "A reconfiguration method for photovoltaic array of stratospheric airship based on multilevel optimization algorithm," Applied Energy, Elsevier, vol. 352(C).
    4. Zhang, Yahui & You, Xiongxiong & Song, Yunfeng & Zhao, Yahui & Wei, Zeyi & Jiao, Xiaohong, 2025. "Hierarchical eco-driving of connected hybrid electric vehicles: Integrating predictive cruise control and cost-to-go approximation-guided energy management," Energy, Elsevier, vol. 319(C).
    5. Fan Wang & Yina Hong & Xiaohuan Zhao, 2025. "Research and Comparative Analysis of Energy Management Strategies for Hybrid Electric Vehicles: A Review," Energies, MDPI, vol. 18(11), pages 1-28, May.
    6. Liu, Yang & Sun, Kangwen & Lv, Mingyun, 2024. "Mission-oriented dynamic reconfiguration of airborne photovoltaic array based on multidisciplinary model," Renewable Energy, Elsevier, vol. 234(C).
    7. Zhao, Yinghua & Huang, Siqi & Wang, Xiaoyu & Shi, Jingwu & Yao, Shouwen, 2024. "Energy management with adaptive moving average filter and deep deterministic policy gradient reinforcement learning for fuel cell hybrid electric vehicles," Energy, Elsevier, vol. 312(C).
    8. Hu, Dong & Huang, Chao & Wu, Jingda & Wei, Henglai & Pi, Dawei, 2025. "Enhancing data-driven energy management strategy via digital expert guidance for electrified vehicles," Applied Energy, Elsevier, vol. 381(C).
    9. Yan Tong & Issam Salhi & Qin Wang & Gang Lu & Shengyu Wu, 2025. "Bidirectional DC-DC Converter Topologies for Hybrid Energy Storage Systems in Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 18(9), pages 1-29, May.
    10. Wang, Zhiguo & Wei, Hongqian & Xi, Yecheng & Xiao, Gongwei, 2024. "Data-driven energy utilization for plug-in hybrid electric bus with driving patten application and battery health considerations," Energy, Elsevier, vol. 310(C).
    11. Li, Jianwei & Liu, Jie & Yang, Qingqing & Wang, Tianci & He, Hongwen & Wang, Hanxiao & Sun, Fengchun, 2025. "Reinforcement learning based energy management for fuel cell hybrid electric vehicles: A comprehensive review on decision process reformulation and strategy implementation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 213(C).
    12. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    13. Na Yeon An & Jung Hyun Yang & Eunyong Song & Sung-Ho Hwang & Hyung-Gi Byun & Sanguk Park, 2024. "Digital Twin-Based Hydrogen Refueling Station (HRS) Safety Model: CNN-Based Decision-Making and 3D Simulation," Sustainability, MDPI, vol. 16(21), pages 1-26, October.
    14. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    15. Su, Chengguo & Li, Li & Zhang, Taiheng & Sui, Quan & Yang, Yunbo, 2025. "A real-time scheduling framework of cascade hydropower-photovoltaic power complementary systems based on model predictive control," Applied Energy, Elsevier, vol. 392(C).
    16. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    17. Dong, Xiao-Jian & Shen, Jia-Ni & Ma, Zi-Feng & He, Yi-Jun, 2025. "Stochastic optimization of integrated electric vehicle charging stations under photovoltaic uncertainty and battery power constraints," Energy, Elsevier, vol. 314(C).
    18. Massimo Sicilia & Davide Cervone & Pierpaolo Polverino & Cesare Pianese, 2024. "Advancements on Lumped Modelling of Membrane Water Content for Real-Time Prognostics and Control of PEMFC," Energies, MDPI, vol. 17(19), pages 1-20, September.
    19. Mounika, Kandi & Bhattacharjee, Ankur, 2025. "Design and experimental validation for performance analysis of non-isolated power converter topologies in fuel cell integrated dynamic load based local energy systems," Energy, Elsevier, vol. 322(C).
    20. Wang, Jun & Tian, Xinyi & Jiang, Mingjun & Lu, Guodong & Fang, Qiansheng & Ji, Jie & Luo, Chenglong, 2025. "Comparison of the photoelectric power by the flexible nonplanar PV modules in different layout and design," Applied Energy, Elsevier, vol. 388(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:324:y:2025:i:c:s0360544225015749. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.