IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i23p9032-d988076.html
   My bibliography  Save this article

Deep Reinforcement Learning-Based Operation of Transmission Battery Storage with Dynamic Thermal Line Rating

Author

Listed:
  • Vadim Avkhimenia

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Matheus Gemignani

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Tim Weis

    (Mechanical Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Petr Musilek

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada
    Applied Cybernetics, University of Hradec Králové, 500 03 Hradec Králové, Czech Republic)

Abstract

It is well known that dynamic thermal line rating has the potential to use power transmission infrastructure more effectively by allowing higher currents when lines are cooler; however, it is not commonly implemented. Some of the barriers to implementation can be mitigated using modern battery energy storage systems. This paper proposes a combination of dynamic thermal line rating and battery use through the application of deep reinforcement learning. In particular, several algorithms based on deep deterministic policy gradient and soft actor critic are examined, in both single- and multi-agent settings. The selected algorithms are used to control battery energy storage systems in a 6-bus test grid. The effects of load and transmissible power forecasting on the convergence of those algorithms are also examined. The soft actor critic algorithm performs best, followed by deep deterministic policy gradient, and their multi-agent versions in the same order. One-step forecasting of the load and ampacity does not provide any significant benefit for predicting battery action.

Suggested Citation

  • Vadim Avkhimenia & Matheus Gemignani & Tim Weis & Petr Musilek, 2022. "Deep Reinforcement Learning-Based Operation of Transmission Battery Storage with Dynamic Thermal Line Rating," Energies, MDPI, vol. 15(23), pages 1-15, November.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:23:p:9032-:d:988076
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/23/9032/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/23/9032/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Holger C. Hesse & Rodrigo Martins & Petr Musilek & Maik Naumann & Cong Nam Truong & Andreas Jossen, 2017. "Economic Optimization of Component Sizing for Residential Battery Storage Systems," Energies, MDPI, vol. 10(7), pages 1-19, June.
    2. Akhtar Hussain & Van-Hai Bui & Hak-Man Kim, 2017. "Impact Analysis of Demand Response Intensity and Energy Storage Size on Operation of Networked Microgrids," Energies, MDPI, vol. 10(7), pages 1-19, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dimitar Bozalakov & Mohannad J. Mnati & Joannes Laveyne & Jan Desmet & Lieven Vandevelde, 2019. "Battery Storage Integration in Voltage Unbalance and Overvoltage Mitigation Control Strategies and Its Impact on the Power Quality," Energies, MDPI, vol. 12(8), pages 1-26, April.
    2. Ghorbanzadeh, Milad & Astaneh, Majid & Golzar, Farzin, 2019. "Long-term degradation based analysis for lithium-ion batteries in off-grid wind-battery renewable energy systems," Energy, Elsevier, vol. 166(C), pages 1194-1206.
    3. Pusceddu, Elian & Zakeri, Behnam & Castagneto Gissey, Giorgio, 2021. "Synergies between energy arbitrage and fast frequency response for battery energy storage systems," Applied Energy, Elsevier, vol. 283(C).
    4. Andrea Mazza & Hamidreza Mirtaheri & Gianfranco Chicco & Angela Russo & Maurizio Fantino, 2019. "Location and Sizing of Battery Energy Storage Units in Low Voltage Distribution Networks," Energies, MDPI, vol. 13(1), pages 1-20, December.
    5. Beuse, Martin & Dirksmeier, Mathias & Steffen, Bjarne & Schmidt, Tobias S., 2020. "Profitability of commercial and industrial photovoltaics and battery projects in South-East-Asia," Applied Energy, Elsevier, vol. 271(C).
    6. Liu, Jia & Chen, Xi & Yang, Hongxing & Li, Yutong, 2020. "Energy storage and management system design optimization for a photovoltaic integrated low-energy building," Energy, Elsevier, vol. 190(C).
    7. Mulleriyawage, U.G.K. & Shen, W.X., 2021. "Impact of demand side management on optimal sizing of residential battery energy storage system," Renewable Energy, Elsevier, vol. 172(C), pages 1250-1266.
    8. Jiang, Yinghua & Kang, Lixia & Liu, Yongzhong, 2019. "A unified model to optimize configuration of battery energy storage systems with multiple types of batteries," Energy, Elsevier, vol. 176(C), pages 552-560.
    9. Nicola Blasuttigh & Simone Negri & Alessandro Massi Pavan & Enrico Tironi, 2023. "Optimal Sizing and Environ-Economic Analysis of PV-BESS Systems for Jointly Acting Renewable Self-Consumers," Energies, MDPI, vol. 16(3), pages 1-25, January.
    10. Wu, Wei & Lin, Boqiang, 2018. "Application value of energy storage in power grid: A special case of China electricity market," Energy, Elsevier, vol. 165(PB), pages 1191-1199.
    11. Daniel Reich & Giovanna Oriti, 2021. "Rightsizing the Design of a Hybrid Microgrid," Energies, MDPI, vol. 14(14), pages 1-22, July.
    12. Warren S. Vaz, 2020. "Multiobjective Optimization of a Residential Grid-Tied Solar System," Sustainability, MDPI, vol. 12(20), pages 1-15, October.
    13. Hector Beltran & Pablo Ayuso & Emilio Pérez, 2020. "Lifetime Expectancy of Li-Ion Batteries used for Residential Solar Storage," Energies, MDPI, vol. 13(3), pages 1-18, January.
    14. Say, Kelvin & Csereklyei, Zsuzsanna & Brown, Felix Gabriel & Wang, Changlong, 2023. "The economics of public transport electrification: A case study from Victoria, Australia," Energy Economics, Elsevier, vol. 120(C).
    15. Juha Koskela & Antti Mutanen & Pertti Järventausta, 2020. "Using Load Forecasting to Control Domestic Battery Energy Storage Systems," Energies, MDPI, vol. 13(15), pages 1-20, August.
    16. Anh-Duc Nguyen & Van-Hai Bui & Akhtar Hussain & Duc-Huy Nguyen & Hak-Man Kim, 2018. "Impact of Demand Response Programs on Optimal Operation of Multi-Microgrid System," Energies, MDPI, vol. 11(6), pages 1-18, June.
    17. Chatzisideris, Marios D. & Ohms, Pernille K. & Espinosa, Nieves & Krebs, Frederik C. & Laurent, Alexis, 2019. "Economic and environmental performances of organic photovoltaics with battery storage for residential self-consumption," Applied Energy, Elsevier, vol. 256(C).
    18. Romain Mannini & Julien Eynard & Stéphane Grieu, 2022. "A Survey of Recent Advances in the Smart Management of Microgrids and Networked Microgrids," Energies, MDPI, vol. 15(19), pages 1-37, September.
    19. Tervo, Eric & Agbim, Kenechi & DeAngelis, Freddy & Hernandez, Jeffrey & Kim, Hye Kyung & Odukomaiya, Adewale, 2018. "An economic analysis of residential photovoltaic systems with lithium ion battery storage in the United States," Renewable and Sustainable Energy Reviews, Elsevier, vol. 94(C), pages 1057-1066.
    20. Shabani, Masoume & Wallin, Fredrik & Dahlquist, Erik & Yan, Jinyue, 2022. "Techno-economic assessment of battery storage integrated into a grid-connected and solar-powered residential building under different battery ageing models," Applied Energy, Elsevier, vol. 318(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:23:p:9032-:d:988076. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.