IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v333y2023ics0306261922018906.html
   My bibliography  Save this article

Automated deep reinforcement learning for real-time scheduling strategy of multi-energy system integrated with post-carbon and direct-air carbon captured system

Author

Listed:
  • Alabi, Tobi Michael
  • Lawrence, Nathan P.
  • Lu, Lin
  • Yang, Zaiyue
  • Bhushan Gopaluni, R.

Abstract

The carbon-capturing process with the aid of CO2 removal technology (CDRT) has been recognised as an alternative and a prominent approach to deep decarbonisation. However, the main hindrance is the enormous energy demand and the economic implication of CDRT if not effectively managed. Hence, a novel deep reinforcement learning agent (DRL), integrated with an automated hyperparameter selection feature, is proposed in this study for the real-time scheduling of a multi-energy system (MES) coupled with CDRT. Post-carbon capture systems (PCCS) and direct-air capture systems (DACS) are considered CDRT. Various possible configurations are evaluated using real-time multi-energy data of a district in Arizona, the United States, and CDRT parameters from manufacturers' catalogues and pilot project documentation. The simulation results validate that an optimised soft-actor critic (SAC) DRL algorithm outperformed the Twin-delayed deep deterministic policy gradient (TD3) algorithm due to its maximum entropy feature. We then trained four (4) SAC DRL agents, equivalent to the number of considered case studies, using optimised hyperparameter values and deployed them in real time for evaluation. The results show that the proposed DRL agent can meet the prosumers' multi-energy demand and schedule the CDRT energy demand economically without specified constraints violation. Also, the proposed DRL agent outperformed rule-based scheduling by 23.65%. However, the configuration with PCCS and solid-sorbent DACS is considered the most suitable configuration with a high CO2 captured-released ratio (CCRR) of 38.54, low CO2 released indicator (CRI) value of 2.53, and a 36.5% reduction in CDR cost due to waste heat utilisation and high absorption capacity of the selected sorbent. However, the adoption of CDRT is not economically viable at the current carbon price. Finally, we showed that CDRT would be attractive at a carbon price of 400-450USD/ton with the provision of tax incentives by the policymakers.

Suggested Citation

  • Alabi, Tobi Michael & Lawrence, Nathan P. & Lu, Lin & Yang, Zaiyue & Bhushan Gopaluni, R., 2023. "Automated deep reinforcement learning for real-time scheduling strategy of multi-energy system integrated with post-carbon and direct-air carbon captured system," Applied Energy, Elsevier, vol. 333(C).
    • Handle: RePEc:eee:appene:v:333:y:2023:i:c:s0306261922018906
    DOI: 10.1016/j.apenergy.2022.120633
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922018906
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.120633?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alabi, Tobi Michael & Lu, Lin & Yang, Zaiyue, 2022. "Data-driven optimal scheduling of multi-energy system virtual power plant (MEVPP) incorporating carbon capture system (CCS), electric vehicle flexibility, and clean energy marketer (CEM) strategy," Applied Energy, Elsevier, vol. 314(C).
    2. Ganesh, Akhil Hannegudda & Xu, Bin, 2022. "A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    3. Liu, Jia & Cao, Sunliang & Chen, Xi & Yang, Hongxing & Peng, Jinqing, 2021. "Energy planning of renewable applications in high-rise residential buildings integrating battery and hydrogen vehicle storage," Applied Energy, Elsevier, vol. 281(C).
    4. Alabi, Tobi Michael & Aghimien, Emmanuel I. & Agbajor, Favour D. & Yang, Zaiyue & Lu, Lin & Adeoye, Adebusola R. & Gopaluni, Bhushan, 2022. "A review on the integrated optimization techniques and machine learning approaches for modeling, prediction, and decision making on integrated energy systems," Renewable Energy, Elsevier, vol. 194(C), pages 822-849.
    5. Alabi, Tobi Michael & Lu, Lin & Yang, Zaiyue, 2021. "A novel multi-objective stochastic risk co-optimization model of a zero-carbon multi-energy system (ZCMES) incorporating energy storage aging model and integrated demand response," Energy, Elsevier, vol. 226(C).
    6. Ming Li & Wenqiang Du & Fuzhong Nian, 2014. "An Adaptive Particle Swarm Optimization Algorithm Based on Directed Weighted Complex Network," Mathematical Problems in Engineering, Hindawi, vol. 2014, pages 1-7, April.
    7. Zhang, Guangming & Wang, Wei & Chen, Zhenyu & Li, Ruilian & Niu, Yuguang, 2022. "Modeling and optimal dispatch of a carbon-cycle integrated energy system for low-carbon and economic operation," Energy, Elsevier, vol. 240(C).
    8. Sarah Deutz & André Bardow, 2021. "Life-cycle assessment of an industrial direct air capture process based on temperature–vacuum swing adsorption," Nature Energy, Nature, vol. 6(2), pages 203-213, February.
    9. Giulia Realmonte & Laurent Drouet & Ajay Gambhir & James Glynn & Adam Hawkes & Alexandre C. Köberle & Massimo Tavoni, 2019. "An inter-model assessment of the role of direct air capture in deep mitigation pathways," Nature Communications, Nature, vol. 10(1), pages 1-12, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhao, Naixin & Gu, Wenbo & Zheng, Zipeng & Ma, Tao, 2023. "Multi-objective bi-level planning of the integrated energy system considering uncertain user loads and carbon emission during the equipment manufacturing process," Renewable Energy, Elsevier, vol. 216(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Song, Yuguang & Xia, Mingchao & Yang, Liu & Chen, Qifang & Su, Su, 2023. "Multi-granularity source-load-storage cooperative dispatch based on combined robust optimization and stochastic optimization for a highway service area micro-energy grid," Renewable Energy, Elsevier, vol. 205(C), pages 747-762.
    2. An, Keju & Farooqui, Azharuddin & McCoy, Sean T., 2022. "The impact of climate on solvent-based direct air capture systems," Applied Energy, Elsevier, vol. 325(C).
    3. Galán-Martín, Ángel & Contreras, María del Mar & Romero, Inmaculada & Ruiz, Encarnación & Bueno-Rodríguez, Salvador & Eliche-Quesada, Dolores & Castro-Galiano, Eulogio, 2022. "The potential role of olive groves to deliver carbon dioxide removal in a carbon-neutral Europe: Opportunities and challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 165(C).
    4. Shu, David Yang & Deutz, Sarah & Winter, Benedikt Alexander & Baumgärtner, Nils & Leenders, Ludger & Bardow, André, 2023. "The role of carbon capture and storage to achieve net-zero energy systems: Trade-offs between economics and the environment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 178(C).
    5. Yang Qiu & Patrick Lamers & Vassilis Daioglou & Noah McQueen & Harmen-Sytze Boer & Mathijs Harmsen & Jennifer Wilcox & André Bardow & Sangwon Suh, 2022. "Environmental trade-offs of direct air capture technologies in climate change mitigation toward 2100," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    6. Motlaghzadeh, Kasra & Schweizer, Vanessa & Craik, Neil & Moreno-Cruz, Juan, 2023. "Key uncertainties behind global projections of direct air capture deployment," Applied Energy, Elsevier, vol. 348(C).
    7. Yicheng Li & Lixiong Xu & Xiangmei Lv & Yiran Xiao, 2022. "Low-Carbon Scheduling of Integrated Electricity and Gas Distribution System Considering V2G," Energies, MDPI, vol. 15(24), pages 1-18, December.
    8. Alabi, Tobi Michael & Lu, Lin & Yang, Zaiyue, 2022. "Data-driven optimal scheduling of multi-energy system virtual power plant (MEVPP) incorporating carbon capture system (CCS), electric vehicle flexibility, and clean energy marketer (CEM) strategy," Applied Energy, Elsevier, vol. 314(C).
    9. Alabi, Tobi Michael & Aghimien, Emmanuel I. & Agbajor, Favour D. & Yang, Zaiyue & Lu, Lin & Adeoye, Adebusola R. & Gopaluni, Bhushan, 2022. "A review on the integrated optimization techniques and machine learning approaches for modeling, prediction, and decision making on integrated energy systems," Renewable Energy, Elsevier, vol. 194(C), pages 822-849.
    10. Selene Cobo & Ángel Galán-Martín & Victor Tulus & Mark A. J. Huijbregts & Gonzalo Guillén-Gosálbez, 2022. "Human and planetary health implications of negative emissions technologies," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    11. Ayami Hayashi & Fuminori Sano & Takashi Homma & Keigo Akimoto, 2023. "Mitigating trade-offs between global food access and net-zero emissions: the potential contribution of direct air carbon capture and storage," Climatic Change, Springer, vol. 176(5), pages 1-19, May.
    12. Zhihan Shi & Weisong Han & Guangming Zhang & Zhiqing Bai & Mingxiang Zhu & Xiaodong Lv, 2022. "Research on Low-Carbon Energy Sharing through the Alliance of Integrated Energy Systems with Multiple Uncertainties," Energies, MDPI, vol. 15(24), pages 1-20, December.
    13. Matteo Acquarone & Claudio Maino & Daniela Misul & Ezio Spessa & Antonio Mastropietro & Luca Sorrentino & Enrico Busto, 2023. "Influence of the Reward Function on the Selection of Reinforcement Learning Agents for Hybrid Electric Vehicles Real-Time Control," Energies, MDPI, vol. 16(6), pages 1-22, March.
    14. Ma, Huan & Sun, Qinghan & Chen, Qun & Zhao, Tian & He, Kelun, 2023. "Exergy-based flexibility cost indicator and spatio-temporal coordination principle of distributed multi-energy systems," Energy, Elsevier, vol. 267(C).
    15. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    16. Ángel Galán-Martín & Daniel Vázquez & Selene Cobo & Niall Dowell & José Antonio Caballero & Gonzalo Guillén-Gosálbez, 2021. "Delaying carbon dioxide removal in the European Union puts climate targets at risk," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
    17. Masood S. Alivand & Omid Mazaheri & Yue Wu & Ali Zavabeti & Andrew J. Christofferson & Nastaran Meftahi & Salvy P. Russo & Geoffrey W. Stevens & Colin A. Scholes & Kathryn A. Mumford, 2022. "Engineered assembly of water-dispersible nanocatalysts enables low-cost and green CO2 capture," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    18. Liu, Jia & Yang, Hongxing & Zhou, Yuekuan, 2021. "Peer-to-peer trading optimizations on net-zero energy communities with energy storage of hydrogen and battery vehicles," Applied Energy, Elsevier, vol. 302(C).
    19. Duncan McLaren & Olaf Corry, 2021. "Clash of Geofutures and the Remaking of Planetary Order: Faultlines underlying Conflicts over Geoengineering Governance," Global Policy, London School of Economics and Political Science, vol. 12(S1), pages 20-33, April.
    20. Xu, Jiazhu & Yi, Yuqin, 2023. "Multi-microgrid low-carbon economy operation strategy considering both source and load uncertainty: A Nash bargaining approach," Energy, Elsevier, vol. 263(PB).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:333:y:2023:i:c:s0306261922018906. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.