IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v18y2025i7p1724-d1623978.html
   My bibliography  Save this article

Reinforcement Learning for Optimizing Renewable Energy Utilization in Buildings: A Review on Applications and Innovations

Author

Listed:
  • Panagiotis Michailidis

    (Center for Research and Technology Hellas, 57001 Thessaloniki, Greece
    Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece
    These authors contributed equally to this work.)

  • Iakovos Michailidis

    (Center for Research and Technology Hellas, 57001 Thessaloniki, Greece
    Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece
    These authors contributed equally to this work.)

  • Elias Kosmatopoulos

    (Center for Research and Technology Hellas, 57001 Thessaloniki, Greece
    Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece
    These authors contributed equally to this work.)

Abstract

The integration of renewable energy systems into modern buildings is essential for enhancing energy efficiency, reducing carbon footprints, and advancing intelligent energy management. However, optimizing RES operations within building energy management systems introduces significant complexity, requiring advanced control strategies. One significant branch of modern control algorithms concerns reinforcement learning, a data-driven strategy capable of dynamically managing renewable energy sources and other energy subsystems under uncertainty and real-time constraints. The current review systematically examines RL-based control strategies applied in BEMS frameworks integrating RES technologies between 2015 and 2025, classifying them by algorithmic approach and evaluating the role of multi-agent and hybrid methods in improving real-time adaptability and occupant comfort. Following a thorough explanation of a rigorous selection process—which targeted the most impactful peer-reviewed publications from the last decade, the paper presents the mathematical concepts of RL and multi-agent RL, along with detailed summaries and summary tables of the integrated works to facilitate quick reference to key findings. For evaluation, the paper examines and outlines the different attributes in the field considering the following: methodologies of RL; agent types; value-action networks; reward functions; baseline control approaches; RES types; BEMS types; and building typologies. Grounded on the findings presented in the evaluation section, the paper offers a structured synthesis of emerging research trends and future directions, identifying the strengths and limitations of RL in energy management.

Suggested Citation

  • Panagiotis Michailidis & Iakovos Michailidis & Elias Kosmatopoulos, 2025. "Reinforcement Learning for Optimizing Renewable Energy Utilization in Buildings: A Review on Applications and Innovations," Energies, MDPI, vol. 18(7), pages 1-40, March.
  • Handle: RePEc:gam:jeners:v:18:y:2025:i:7:p:1724-:d:1623978
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/18/7/1724/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/18/7/1724/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Qiu, Dawei & Xue, Juxing & Zhang, Tingqi & Wang, Jianhong & Sun, Mingyang, 2023. "Federated reinforcement learning for smart building joint peer-to-peer energy and carbon allowance trading," Applied Energy, Elsevier, vol. 333(C).
    2. Gao, Yuan & Matsunami, Yuki & Miyata, Shohei & Akashi, Yasunori, 2022. "Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system," Applied Energy, Elsevier, vol. 326(C).
    3. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    4. Pergantis, Elias N. & Priyadarshan, & Theeb, Nadah Al & Dhillon, Parveen & Ore, Jonathan P. & Ziviani, Davide & Groll, Eckhard A. & Kircher, Kevin J., 2024. "Field demonstration of predictive heating control for an all-electric house in a cold climate," Applied Energy, Elsevier, vol. 360(C).
    5. Vennapusa Jagadeeswara Reddy & N. P. Hariram & Mohd Fairusham Ghazali & Sudhakar Kumarasamy, 2024. "Pathway to Sustainability: An Overview of Renewable Energy Integration in Building Systems," Sustainability, MDPI, vol. 16(2), pages 1-36, January.
    6. Yang, Lei & Nagy, Zoltan & Goffin, Philippe & Schlueter, Arno, 2015. "Reinforcement learning for optimal control of low exergy buildings," Applied Energy, Elsevier, vol. 156(C), pages 577-586.
    7. Langer, Lissy & Volling, Thomas, 2022. "A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems," Applied Energy, Elsevier, vol. 327(C).
    8. Kazmi, H. & D’Oca, S. & Delmastro, C. & Lodeweyckx, S. & Corgnati, S.P., 2016. "Generalizable occupant-driven optimization model for domestic hot water production in NZEB," Applied Energy, Elsevier, vol. 175(C), pages 1-15.
    9. Gaviria, Jorge Felipe & Narváez, Gabriel & Guillen, Camilo & Giraldo, Luis Felipe & Bressan, Michael, 2022. "Machine learning in photovoltaic systems: A review," Renewable Energy, Elsevier, vol. 196(C), pages 298-318.
    10. Touzani, Samir & Prakash, Anand Krishnan & Wang, Zhe & Agarwal, Shreya & Pritoni, Marco & Kiran, Mariam & Brown, Richard & Granderson, Jessica, 2021. "Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency," Applied Energy, Elsevier, vol. 304(C).
    11. Anvari-Moghaddam, Amjad & Rahimi-Kian, Ashkan & Mirian, Maryam S. & Guerrero, Josep M., 2017. "A multi-agent based energy management solution for integrated buildings and microgrid system," Applied Energy, Elsevier, vol. 203(C), pages 41-56.
    12. Yujian Ye & Dawei Qiu & Huiyu Wang & Yi Tang & Goran Strbac, 2021. "Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning," Energies, MDPI, vol. 14(3), pages 1-22, January.
    13. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    14. Sunyong Kim & Hyuk Lim, 2018. "Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings," Energies, MDPI, vol. 11(8), pages 1-19, August.
    15. Wang, Zixuan & Xiao, Fu & Ran, Yi & Li, Yanxue & Xu, Yang, 2024. "Scalable energy management approach of residential hybrid energy system using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 367(C).
    16. Deng, Xiangtian & Zhang, Yi & Jiang, Yi & Zhang, Yi & Qi, He, 2024. "A novel operation method for renewable building by combining distributed DC energy system and deep reinforcement learning," Applied Energy, Elsevier, vol. 353(PB).
    17. Wang, Zhe & Hong, Tianzhen, 2020. "Reinforcement learning for building controls: The opportunities and challenges," Applied Energy, Elsevier, vol. 269(C).
    18. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    19. Drgoňa, Ján & Picard, Damien & Kvasnica, Michal & Helsen, Lieve, 2018. "Approximate model predictive building control via machine learning," Applied Energy, Elsevier, vol. 218(C), pages 199-216.
    20. Soteris A. Kalogirou, 2015. "Building integration of solar renewable energy systems towards zero or nearly zero energy buildings," International Journal of Low-Carbon Technologies, Oxford University Press, vol. 10(4), pages 379-385.
    21. Pinto, Giuseppe & Kathirgamanathan, Anjukan & Mangina, Eleni & Finn, Donal P. & Capozzoli, Alfonso, 2022. "Enhancing energy management in grid-interactive buildings: A comparison among cooperative and coordinated architectures," Applied Energy, Elsevier, vol. 310(C).
    22. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    23. Gao, Yuan & Matsunami, Yuki & Miyata, Shohei & Akashi, Yasunori, 2022. "Operational optimization for off-grid renewable building energy system using deep reinforcement learning," Applied Energy, Elsevier, vol. 325(C).
    24. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    25. Panagiotis Michailidis & Iakovos Michailidis & Socratis Gkelios & Elias Kosmatopoulos, 2024. "Artificial Neural Network Applications for Energy Management in Buildings: Current Trends and Future Directions," Energies, MDPI, vol. 17(3), pages 1-47, January.
    26. Laura Canale & Anna Rita Di Fazio & Mario Russo & Andrea Frattolillo & Marco Dell’Isola, 2021. "An Overview on Functional Integration of Hybrid Renewable Energy Systems in Multi-Energy Buildings," Energies, MDPI, vol. 14(4), pages 1-33, February.
    27. Heidari, Amirreza & Maréchal, François & Khovalyg, Dolaana, 2022. "Reinforcement Learning for proactive operation of residential energy systems by learning stochastic occupant behavior and fluctuating solar energy: Balancing comfort, hygiene and energy use," Applied Energy, Elsevier, vol. 318(C).
    28. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    29. Iakovos T. Michailidis & Roozbeh Sangi & Panagiotis Michailidis & Thomas Schild & Johannes Fuetterer & Dirk Mueller & Elias B. Kosmatopoulos, 2020. "Balancing Energy Efficiency with Indoor Comfort Using Smart Control Agents: A Simulative Case Study," Energies, MDPI, vol. 13(23), pages 1-28, November.
    30. Kofinas, P. & Dounis, A.I. & Vouros, G.A., 2018. "Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Applied Energy, Elsevier, vol. 219(C), pages 53-67.
    31. Davide Deltetto & Davide Coraci & Giuseppe Pinto & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "Exploring the Potentialities of Deep Reinforcement Learning for Incentive-Based Demand Response in a Cluster of Small Commercial Buildings," Energies, MDPI, vol. 14(10), pages 1-25, May.
    32. Xie, Jiahan & Ajagekar, Akshay & You, Fengqi, 2023. "Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings," Applied Energy, Elsevier, vol. 342(C).
    33. Jung, Seunghoon & Jeoung, Jaewon & Kang, Hyuna & Hong, Taehoon, 2021. "Optimal planning of a rooftop PV system using GIS-based reinforcement learning," Applied Energy, Elsevier, vol. 298(C).
    34. Zia, Muhammad Fahad & Elbouchikhi, Elhoussin & Benbouzid, Mohamed, 2018. "Microgrids energy management systems: A critical review on methods, solutions, and prospects," Applied Energy, Elsevier, vol. 222(C), pages 1033-1055.
    35. Shen, Rendong & Zhong, Shengyuan & Wen, Xin & An, Qingsong & Zheng, Ruifan & Li, Yang & Zhao, Jun, 2022. "Multi-agent deep reinforcement learning optimization framework for building energy system with renewable energy," Applied Energy, Elsevier, vol. 312(C).
    36. Michailidis, Iakovos T. & Schild, Thomas & Sangi, Roozbeh & Michailidis, Panagiotis & Korkas, Christos & Fütterer, Johannes & Müller, Dirk & Kosmatopoulos, Elias B., 2018. "Energy-efficient HVAC management using cooperative, self-trained, control agents: A real-life German building case study," Applied Energy, Elsevier, vol. 211(C), pages 113-125.
    37. Tomin, Nikita & Shakirov, Vladislav & Kozlov, Aleksander & Sidorov, Denis & Kurbatsky, Victor & Rehtanz, Christian & Lora, Electo E.S., 2022. "Design and optimal energy management of community microgrids with flexible renewable energy sources," Renewable Energy, Elsevier, vol. 183(C), pages 903-921.
    38. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    2. Ayas Shaqour & Aya Hagishima, 2022. "Systematic Review on Deep Reinforcement Learning-Based Energy Management for Different Building Types," Energies, MDPI, vol. 15(22), pages 1-27, November.
    3. Gao, Yuan & Matsunami, Yuki & Miyata, Shohei & Akashi, Yasunori, 2022. "Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system," Applied Energy, Elsevier, vol. 326(C).
    4. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    5. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    6. Wenya Xu & Yanxue Li & Guanjie He & Yang Xu & Weijun Gao, 2023. "Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control," Energies, MDPI, vol. 16(13), pages 1-19, June.
    7. Charalampos Rafail Lazaridis & Iakovos Michailidis & Georgios Karatzinis & Panagiotis Michailidis & Elias Kosmatopoulos, 2024. "Evaluating Reinforcement Learning Algorithms in Residential Energy Saving and Comfort Management," Energies, MDPI, vol. 17(3), pages 1-33, January.
    8. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    9. Pinto, Giuseppe & Kathirgamanathan, Anjukan & Mangina, Eleni & Finn, Donal P. & Capozzoli, Alfonso, 2022. "Enhancing energy management in grid-interactive buildings: A comparison among cooperative and coordinated architectures," Applied Energy, Elsevier, vol. 310(C).
    10. Zhou, Xinlei & Du, Han & Xue, Shan & Ma, Zhenjun, 2024. "Recent advances in data mining and machine learning for enhanced building energy management," Energy, Elsevier, vol. 307(C).
    11. Schmitz, Simon & Brucke, Karoline & Kasturi, Pranay & Ansari, Esmail & Klement, Peter, 2024. "Forecast-based and data-driven reinforcement learning for residential heat pump operation," Applied Energy, Elsevier, vol. 371(C).
    12. Pinthurat, Watcharakorn & Surinkaew, Tossaporn & Hredzak, Branislav, 2024. "An overview of reinforcement learning-based approaches for smart home energy management systems with energy storages," Renewable and Sustainable Energy Reviews, Elsevier, vol. 202(C).
    13. Coraci, Davide & Brandi, Silvio & Hong, Tianzhen & Capozzoli, Alfonso, 2023. "Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings," Applied Energy, Elsevier, vol. 333(C).
    14. Soleimanzade, Mohammad Amin & Kumar, Amit & Sadrzadeh, Mohtada, 2022. "Novel data-driven energy management of a hybrid photovoltaic-reverse osmosis desalination system using deep reinforcement learning," Applied Energy, Elsevier, vol. 317(C).
    15. Dominik Latoń & Jakub Grela & Andrzej Ożadowicz, 2024. "Applications of Deep Reinforcement Learning for Home Energy Management Systems: A Review," Energies, MDPI, vol. 17(24), pages 1-30, December.
    16. Li, Yanxue & Wang, Zixuan & Xu, Wenya & Gao, Weijun & Xu, Yang & Xiao, Fu, 2023. "Modeling and energy dynamic control for a ZEH via hybrid model-based deep reinforcement learning," Energy, Elsevier, vol. 277(C).
    17. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    18. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    19. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    20. Wang, Zixuan & Xiao, Fu & Ran, Yi & Li, Yanxue & Xu, Yang, 2024. "Scalable energy management approach of residential hybrid energy system using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 367(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:18:y:2025:i:7:p:1724-:d:1623978. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.