IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v262y2020ics0306261920300921.html
   My bibliography  Save this article

Introducing reinforcement learning to the energy system design process

Author

Listed:
  • Perera, A.T.D.
  • Wickramasinghe, P.U.
  • Nik, Vahid M.
  • Scartezzini, Jean-Louis

Abstract

Design optimization of distributed energy systems has become an interest of a wider group of researchers due the capability of these systems to integrate non-dispatchable renewable energy technologies such as solar PV and wind. White box models, using linear and mixed integer linear programing techniques, are often used in their design. However, the increased complexity of energy flow (especially due to cyber-physical interactions) and uncertainties challenge the application of white box models. This is where data driven methodologies become effective, as they demonstrate higher flexibility to adapt to different environments, which enables their use for energy planning at regional and national scale.

Suggested Citation

  • Perera, A.T.D. & Wickramasinghe, P.U. & Nik, Vahid M. & Scartezzini, Jean-Louis, 2020. "Introducing reinforcement learning to the energy system design process," Applied Energy, Elsevier, vol. 262(C).
    • Handle: RePEc:eee:appene:v:262:y:2020:i:c:s0306261920300921
    DOI: 10.1016/j.apenergy.2020.114580
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920300921
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2020.114580?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Perera, A.T.D. & Nik, Vahid M. & Mauree, Dasaraden & Scartezzini, Jean-Louis, 2017. "Electrical hubs: An effective way to integrate non-dispatchable renewable energy sources with minimum impact to the grid," Applied Energy, Elsevier, vol. 190(C), pages 232-248.
    2. Gabrielli, Paolo & Gazzani, Matteo & Martelli, Emanuele & Mazzotti, Marco, 2018. "Optimal design of multi-energy systems with seasonal storage," Applied Energy, Elsevier, vol. 219(C), pages 408-424.
    3. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    4. Salom, Jaume & Marszal, Anna Joanna & Widén, Joakim & Candanedo, José & Lindberg, Karen Byskov, 2014. "Analysis of load match and grid interaction indicators in net zero energy buildings with simulated and monitored data," Applied Energy, Elsevier, vol. 136(C), pages 119-131.
    5. Orehounig, Kristina & Evins, Ralph & Dorer, Viktor, 2015. "Integration of decentralized energy systems in neighbourhoods using the energy hub approach," Applied Energy, Elsevier, vol. 154(C), pages 277-289.
    6. Mauree, Dasaraden & Naboni, Emanuele & Coccolo, Silvia & Perera, A.T.D. & Nik, Vahid M. & Scartezzini, Jean-Louis, 2019. "A review of assessment methods for the urban environment and its energy sustainability to guarantee climate adaptation of future cities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 112(C), pages 733-746.
    7. Nik, Vahid M., 2016. "Making energy simulation easier for future climate – Synthesizing typical and extreme weather data sets out of regional climate models (RCMs)," Applied Energy, Elsevier, vol. 177(C), pages 204-226.
    8. Mohammadi, Mohammad & Noorollahi, Younes & Mohammadi-ivatloo, Behnam & Yousefi, Hossein, 2017. "Energy hub: From a model to a concept – A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 80(C), pages 1512-1527.
    9. Perera, A.T.D. & Wickremasinghe, D.M.I.J. & Mahindarathna, D.V.S. & Attalage, R.A. & Perera, K.K.C.K. & Bartholameuz, E.M., 2012. "Sensitivity of internal combustion generator capacity in standalone hybrid energy systems," Energy, Elsevier, vol. 39(1), pages 403-411.
    10. Perera, A.T.D. & Wickramasinghe, P.U. & Nik, Vahid M. & Scartezzini, Jean-Louis, 2019. "Machine learning methods to assist energy system optimization," Applied Energy, Elsevier, vol. 243(C), pages 191-205.
    11. Perera, A.T.D. & Attalage, R.A. & Perera, K.K.C.K. & Dassanayake, V.P.C., 2013. "Designing standalone hybrid energy systems minimizing initial investment, life cycle cost and pollutant emission," Energy, Elsevier, vol. 54(C), pages 220-230.
    12. Dasaraden Mauree & Silvia Coccolo & Amarasinghage Tharindu Dasun Perera & Vahid Nik & Jean-Louis Scartezzini & Emanuele Naboni, 2018. "A New Framework to Evaluate Urban Design Using Urban Microclimatic Modeling in Future Climatic Conditions," Sustainability, MDPI, vol. 10(4), pages 1-20, April.
    13. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    14. Murray, Portia & Orehounig, Kristina & Grosspietsch, David & Carmeliet, Jan, 2018. "A comparison of storage systems in neighbourhood decentralized energy system applications from 2015 to 2050," Applied Energy, Elsevier, vol. 231(C), pages 1285-1306.
    15. Perera, A.T.D. & Nik, Vahid M. & Mauree, Dasaraden & Scartezzini, Jean-Louis, 2017. "An integrated approach to design site specific distributed electrical hubs combining optimization, multi-criterion assessment and decision making," Energy, Elsevier, vol. 134(C), pages 103-120.
    16. Jayasekara, Saliya & Halgamuge, Saman K. & Attalage, Rahula A. & Rajarathne, Rohitha, 2014. "Optimum sizing and tracking of combined cooling heating and power systems for bulk energy consumers," Applied Energy, Elsevier, vol. 118(C), pages 124-134.
    17. Evins, Ralph, 2015. "Multi-level optimization of building design, energy system sizing and operation," Energy, Elsevier, vol. 90(P2), pages 1775-1789.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shan-Jen Cheng & Wen-Ken Li & Te-Jen Chang & Chang-Hung Hsu, 2021. "Data-Driven Prognostics of the SOFC System Based on Dynamic Neural Network Models," Energies, MDPI, vol. 14(18), pages 1-17, September.
    2. Nik, Vahid M. & Hosseini, Mohammad, 2023. "CIRLEM: a synergic integration of Collective Intelligence and Reinforcement learning in Energy Management for enhanced climate resilience and lightweight computation," Applied Energy, Elsevier, vol. 350(C).
    3. Wang, Zhengchao & Perera, A.T.D., 2020. "Integrated platform to design robust energy internet," Applied Energy, Elsevier, vol. 269(C).
    4. Nik, Vahid M. & Moazami, Amin, 2021. "Using collective intelligence to enhance demand flexibility and climate resilience in urban areas," Applied Energy, Elsevier, vol. 281(C).
    5. Mazzeo, Domenico & Herdem, Münür Sacit & Matera, Nicoletta & Bonini, Matteo & Wen, John Z. & Nathwani, Jatin & Oliveti, Giuseppe, 2021. "Artificial intelligence application for the performance prediction of a clean energy community," Energy, Elsevier, vol. 232(C).
    6. Perera, A.T.D. & Khayatian, F. & Eggimann, S. & Orehounig, K. & Halgamuge, Saman, 2022. "Quantifying the climate and human-system-driven uncertainties in energy planning by using GANs," Applied Energy, Elsevier, vol. 328(C).
    7. Perera, A.T.D. & Javanroodi, Kavan & Nik, Vahid M., 2021. "Climate resilient interconnected infrastructure: Co-optimization of energy systems and urban morphology," Applied Energy, Elsevier, vol. 285(C).
    8. Andrew Chapman, 2023. "Enhancing Survey Efficiency and Predictive Ability in Energy System Design through Machine Learning: A Workflow-Based Approach for Improved Outcomes," Energies, MDPI, vol. 16(13), pages 1-16, June.
    9. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    10. Zheng, Lingwei & Wu, Hao & Guo, Siqi & Sun, Xinyu, 2023. "Real-time dispatch of an integrated energy system based on multi-stage reinforcement learning with an improved action-choosing strategy," Energy, Elsevier, vol. 277(C).
    11. Perera, A.T.D. & Zhao, Bingyu & Wang, Zhe & Soga, Kenichi & Hong, Tianzhen, 2023. "Optimal design of microgrids to improve wildfire resilience for vulnerable communities at the wildland-urban interface," Applied Energy, Elsevier, vol. 335(C).
    12. Kalina, Jacek, 2023. "The quest for game changers - Review of new trends and innovations in the design of large-scale energy systems," Energy, Elsevier, vol. 277(C).
    13. Warut Pannakkong & Vu Thanh Vinh & Nguyen Ngoc Minh Tuyen & Jirachai Buddhakulsomsiri, 2023. "A Reinforcement Learning Approach for Ensemble Machine Learning Models in Peak Electricity Forecasting," Energies, MDPI, vol. 16(13), pages 1-20, July.
    14. Kiani-Moghaddam, Mohammad & Soltani, Mohsen N. & Kalogirou, Soteris A. & Mahian, Omid & Arabkoohsar, Ahmad, 2023. "A review of neighborhood level multi-carrier energy hubs—uncertainty and problem-solving process," Energy, Elsevier, vol. 281(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    2. Perera, A.T.D. & Wickramasinghe, P.U. & Nik, Vahid M. & Scartezzini, Jean-Louis, 2019. "Machine learning methods to assist energy system optimization," Applied Energy, Elsevier, vol. 243(C), pages 191-205.
    3. Perera, A.T.D. & Javanroodi, Kavan & Nik, Vahid M., 2021. "Climate resilient interconnected infrastructure: Co-optimization of energy systems and urban morphology," Applied Energy, Elsevier, vol. 285(C).
    4. Mauree, Dasaraden & Naboni, Emanuele & Coccolo, Silvia & Perera, A.T.D. & Nik, Vahid M. & Scartezzini, Jean-Louis, 2019. "A review of assessment methods for the urban environment and its energy sustainability to guarantee climate adaptation of future cities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 112(C), pages 733-746.
    5. Perera, A.T.D. & Nik, Vahid M. & Wickramasinghe, P.U. & Scartezzini, Jean-Louis, 2019. "Redefining energy system flexibility for distributed energy system design," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    6. Perera, A.T.D. & Soga, Kenichi & Xu, Yujie & Nico, Peter S. & Hong, Tianzhen, 2023. "Enhancing flexibility for climate change using seasonal energy storage (aquifer thermal energy storage) in distributed energy systems," Applied Energy, Elsevier, vol. 340(C).
    7. Perera, A.T.D. & Nik, Vahid M. & Mauree, Dasaraden & Scartezzini, Jean-Louis, 2017. "Electrical hubs: An effective way to integrate non-dispatchable renewable energy sources with minimum impact to the grid," Applied Energy, Elsevier, vol. 190(C), pages 232-248.
    8. Perera, A.T.D. & Nik, Vahid M. & Mauree, Dasaraden & Scartezzini, Jean-Louis, 2017. "An integrated approach to design site specific distributed electrical hubs combining optimization, multi-criterion assessment and decision making," Energy, Elsevier, vol. 134(C), pages 103-120.
    9. Perera, A.T.D. & Coccolo, Silvia & Scartezzini, Jean-Louis & Mauree, Dasaraden, 2018. "Quantifying the impact of urban climate by extending the boundaries of urban energy system modeling," Applied Energy, Elsevier, vol. 222(C), pages 847-860.
    10. Karni Siraganyan & Amarasinghage Tharindu Dasun Perera & Jean-Louis Scartezzini & Dasaraden Mauree, 2019. "Eco-Sim: A Parametric Tool to Evaluate the Environmental and Economic Feasibility of Decentralized Energy Systems," Energies, MDPI, vol. 12(5), pages 1-22, February.
    11. Perera, A.T.D. & Zhao, Bingyu & Wang, Zhe & Soga, Kenichi & Hong, Tianzhen, 2023. "Optimal design of microgrids to improve wildfire resilience for vulnerable communities at the wildland-urban interface," Applied Energy, Elsevier, vol. 335(C).
    12. Perera, A.T.D. & Hong, Tianzhen, 2023. "Vulnerability and resilience of urban energy ecosystems to extreme climate events: A systematic review and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).
    13. Qiu, Dawei & Dong, Zihang & Zhang, Xi & Wang, Yi & Strbac, Goran, 2022. "Safe reinforcement learning for real-time automatic control in a smart energy-hub," Applied Energy, Elsevier, vol. 309(C).
    14. Nik, Vahid M. & Moazami, Amin, 2021. "Using collective intelligence to enhance demand flexibility and climate resilience in urban areas," Applied Energy, Elsevier, vol. 281(C).
    15. Waibel, Christoph & Evins, Ralph & Carmeliet, Jan, 2019. "Co-simulation and optimization of building geometry and multi-energy systems: Interdependencies in energy supply, energy demand and solar potentials," Applied Energy, Elsevier, vol. 242(C), pages 1661-1682.
    16. Els van der Roest & Theo Fens & Martin Bloemendal & Stijn Beernink & Jan Peter van der Hoek & Ad J. M. van Wijk, 2021. "The Impact of System Integration on System Costs of a Neighborhood Energy and Water System," Energies, MDPI, vol. 14(9), pages 1-33, May.
    17. Leprince, Julien & Schledorn, Amos & Guericke, Daniela & Dominkovic, Dominik Franjo & Madsen, Henrik & Zeiler, Wim, 2023. "Can occupant behaviors affect urban energy planning? Distributed stochastic optimization for energy communities," Applied Energy, Elsevier, vol. 348(C).
    18. Dasaraden Mauree & Silvia Coccolo & Amarasinghage Tharindu Dasun Perera & Vahid Nik & Jean-Louis Scartezzini & Emanuele Naboni, 2018. "A New Framework to Evaluate Urban Design Using Urban Microclimatic Modeling in Future Climatic Conditions," Sustainability, MDPI, vol. 10(4), pages 1-20, April.
    19. van der Roest, Els & Snip, Laura & Fens, Theo & van Wijk, Ad, 2020. "Introducing Power-to-H3: Combining renewable electricity with heat, water and hydrogen production and storage in a neighbourhood," Applied Energy, Elsevier, vol. 257(C).
    20. Mavromatidis, Georgios & Petkov, Ivalin, 2021. "MANGO: A novel optimization model for the long-term, multi-stage planning of decentralized multi-energy systems," Applied Energy, Elsevier, vol. 288(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:262:y:2020:i:c:s0306261920300921. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.