IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v11y2018i8p2010-d161526.html
   My bibliography  Save this article

Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings

Author

Listed:
  • Sunyong Kim

    (School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Korea)

  • Hyuk Lim

    (School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Korea)

Abstract

A smart grid facilitates more effective energy management of an electrical grid system. Because both energy consumption and associated building operation costs are increasing rapidly around the world, the need for flexible and cost-effective management of the energy used by buildings in a smart grid environment is increasing. In this paper, we consider an energy management system for a smart energy building connected to an external grid (utility) as well as distributed energy resources including a renewable energy source, energy storage system, and vehicle-to-grid station. First, the energy management system is modeled using a Markov decision process that completely describes the state, action, transition probability, and rewards of the system. Subsequently, a reinforcement-learning-based energy management algorithm is proposed to reduce the operation energy costs of the target smart energy building under unknown future information. The results of numerical simulation based on the data measured in real environments show that the proposed energy management algorithm gradually reduces energy costs via learning processes compared to other random and non-learning-based algorithms.

Suggested Citation

  • Sunyong Kim & Hyuk Lim, 2018. "Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings," Energies, MDPI, vol. 11(8), pages 1-19, August.
    • Handle: RePEc:gam:jeners:v:11:y:2018:i:8:p:2010-:d:161526
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/11/8/2010/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/11/8/2010/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Honarmand, Masoud & Zakariazadeh, Alireza & Jadid, Shahram, 2014. "Optimal scheduling of electric vehicles in an intelligent parking lot considering vehicle-to-grid concept and battery condition," Energy, Elsevier, vol. 65(C), pages 572-579.
    2. Burger, Scott P. & Luke, Max, 2017. "Business models for distributed energy resources: A review and empirical analysis," Energy Policy, Elsevier, vol. 109(C), pages 230-248.
    3. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bartlomiej Kawa & Piotr Borkowski, 2023. "Integration of Machine Learning Solutions in the Building Automation System," Energies, MDPI, vol. 16(11), pages 1-18, June.
    2. Dorokhova, Marina & Martinson, Yann & Ballif, Christophe & Wyrsch, Nicolas, 2021. "Deep reinforcement learning control of electric vehicle charging in the presence of photovoltaic generation," Applied Energy, Elsevier, vol. 301(C).
    3. Harri Aaltonen & Seppo Sierla & Rakshith Subramanya & Valeriy Vyatkin, 2021. "A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage," Energies, MDPI, vol. 14(17), pages 1-20, September.
    4. Alqahtani, Mohammed & Hu, Mengqi, 2022. "Dynamic energy scheduling and routing of multiple electric vehicles using deep reinforcement learning," Energy, Elsevier, vol. 244(PA).
    5. Yujian Ye & Dawei Qiu & Huiyu Wang & Yi Tang & Goran Strbac, 2021. "Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning," Energies, MDPI, vol. 14(3), pages 1-22, January.
    6. Wei Xu & Xiaoli Zhang & Anbang Peng & Yue Liang, 2020. "Deep Reinforcement Learning for Cascaded Hydropower Reservoirs Considering Inflow Forecasts," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 34(9), pages 3003-3018, July.
    7. Ki-Beom Lee & Mohamed A. Ahmed & Dong-Ki Kang & Young-Chon Kim, 2020. "Deep Reinforcement Learning Based Optimal Route and Charging Station Selection," Energies, MDPI, vol. 13(23), pages 1-22, November.
    8. Khalil Bachiri & Ali Yahyaouy & Hamid Gualous & Maria Malek & Younes Bennani & Philippe Makany & Nicoleta Rogovschi, 2023. "Multi-Agent DDPG Based Electric Vehicles Charging Station Recommendation," Energies, MDPI, vol. 16(16), pages 1-17, August.
    9. Khawaja Haider Ali & Marvin Sigalo & Saptarshi Das & Enrico Anderlini & Asif Ali Tahir & Mohammad Abusara, 2021. "Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation," Energies, MDPI, vol. 14(18), pages 1-18, September.
    10. Ahmed M. Abed & Ali AlArjani, 2022. "The Neural Network Classifier Works Efficiently on Searching in DQN Using the Autonomous Internet of Things Hybridized by the Metaheuristic Techniques to Reduce the EVs’ Service Scheduling Time," Energies, MDPI, vol. 15(19), pages 1-25, September.
    11. Svetozarevic, B. & Baumann, C. & Muntwiler, S. & Di Natale, L. & Zeilinger, M.N. & Heer, P., 2022. "Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments," Applied Energy, Elsevier, vol. 307(C).
    12. Van-Hai Bui & Akhtar Hussain & Hak-Man Kim, 2019. "Q-Learning-Based Operation Strategy for Community Battery Energy Storage System (CBESS) in Microgrid System," Energies, MDPI, vol. 12(9), pages 1-17, May.
    13. Grace Muriithi & Sunetra Chowdhury, 2021. "Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach," Energies, MDPI, vol. 14(9), pages 1-24, May.
    14. Khawaja Haider Ali & Mohammad Abusara & Asif Ali Tahir & Saptarshi Das, 2023. "Dual-Layer Q-Learning Strategy for Energy Management of Battery Storage in Grid-Connected Microgrids," Energies, MDPI, vol. 16(3), pages 1-17, January.
    15. Bio Gassi, Karim & Baysal, Mustafa, 2023. "Improving real-time energy decision-making model with an actor-critic agent in modern microgrids with energy storage devices," Energy, Elsevier, vol. 263(PE).
    16. Aguilar, J. & Garces-Jimenez, A. & R-Moreno, M.D. & García, Rodrigo, 2021. "A systematic literature review on the use of artificial intelligence in energy self-management in smart buildings," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    17. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    18. Jaehong Whang & Woohyun Hwang & Yeuntae Yoo & Gilsoo Jang, 2018. "Introduction of Smart Grid Station Configuration and Application in Guri Branch Office of KEPCO," Sustainability, MDPI, vol. 10(10), pages 1-18, September.
    19. Lilia Tightiz & Joon Yoo, 2022. "A Review on a Data-Driven Microgrid Management System Integrating an Active Distribution Network: Challenges, Issues, and New Trends," Energies, MDPI, vol. 15(22), pages 1-24, November.
    20. Ying Ji & Jianhui Wang & Jiacan Xu & Xiaoke Fang & Huaguang Zhang, 2019. "Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning," Energies, MDPI, vol. 12(12), pages 1-21, June.
    21. Kang, Hyuna & Jung, Seunghoon & Lee, Minhyun & Hong, Taehoon, 2022. "How to better share energy towards a carbon-neutral city? A review on application strategies of battery energy storage system in city," Renewable and Sustainable Energy Reviews, Elsevier, vol. 157(C).
    22. Ritu Kandari & Neeraj Neeraj & Alexander Micallef, 2022. "Review on Recent Strategies for Integrating Energy Storage Systems in Microgrids," Energies, MDPI, vol. 16(1), pages 1-24, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    2. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    3. Hao Liang & Weihua Zhuang, 2014. "Stochastic Modeling and Optimization in a Microgrid: A Survey," Energies, MDPI, vol. 7(4), pages 1-24, March.
    4. Karlson Hargroves & Benjamin James & Joshua Lane & Peter Newman, 2023. "The Role of Distributed Energy Resources and Associated Business Models in the Decentralised Energy Transition: A Review," Energies, MDPI, vol. 16(10), pages 1-15, May.
    5. Raja S, Charles & Kumar N M, Vijaya & J, Senthil kumar & Nesamalar J, Jeslin Drusila, 2021. "Enhancing system reliability by optimally integrating PHEV charging station and renewable distributed generators: A Bi-Level programming approach," Energy, Elsevier, vol. 229(C).
    6. Bhowmik, Chiranjib & Bhowmik, Sumit & Ray, Amitava & Pandey, Krishna Murari, 2017. "Optimal green energy planning for sustainable development: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 71(C), pages 796-813.
    7. Du, Jiuyu & Chen, Jingfu & Song, Ziyou & Gao, Mingming & Ouyang, Minggao, 2017. "Design method of a power management strategy for variable battery capacities range-extended electric vehicles to improve energy efficiency and cost-effectiveness," Energy, Elsevier, vol. 121(C), pages 32-42.
    8. Miguel Carpintero-Rentería & David Santos-Martín & Josep M. Guerrero, 2019. "Microgrids Literature Review through a Layers Structure," Energies, MDPI, vol. 12(22), pages 1-22, November.
    9. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    10. Stergios Statharas & Yannis Moysoglou & Pelopidas Siskos & Pantelis Capros, 2021. "Simulating the Evolution of Business Models for Electricity Recharging Infrastructure Development by 2030: A Case Study for Greece," Energies, MDPI, vol. 14(9), pages 1-24, April.
    11. Zhou, Hou Sheng & Passey, Rob & Bruce, Anna & Sproul, Alistair B., 2021. "A case study on the behaviour of residential battery energy storage systems during network demand peaks," Renewable Energy, Elsevier, vol. 180(C), pages 712-724.
    12. Waseem Akram & Muaz Niazi & Laszlo Barna Iantovics & Athanasios V. Vasilakos, 2019. "Towards Agent-Based Model Specification of Smart Grid: A Cognitive Agent-Based Computing Approach," Interdisciplinary Description of Complex Systems - scientific journal, Croatian Interdisciplinary Society Provider Homepage: http://indecs.eu, vol. 17(3-B), pages 546-585.
    13. García-Triviño, Pablo & Torreglosa, Juan P. & Fernández-Ramírez, Luis M. & Jurado, Francisco, 2016. "Control and operation of power sources in a medium-voltage direct-current microgrid for an electric vehicle fast charging station with a photovoltaic and a battery energy storage system," Energy, Elsevier, vol. 115(P1), pages 38-48.
    14. Bhandari, Vivek & Sun, Kaiyang & Homans, Frances, 2018. "The profitability of vehicle to grid for system participants - A case study from the Electricity Reliability Council of Texas," Energy, Elsevier, vol. 153(C), pages 278-286.
    15. Boukettaya, Ghada & Krichen, Lotfi, 2014. "A dynamic power management strategy of a grid connected hybrid generation system using wind, photovoltaic and Flywheel Energy Storage System in residential applications," Energy, Elsevier, vol. 71(C), pages 148-159.
    16. Nor Liza Tumeran & Siti Hajar Yusoff & Teddy Surya Gunawan & Mohd Shahrin Abu Hanifah & Suriza Ahmad Zabidi & Bernardi Pranggono & Muhammad Sharir Fathullah Mohd Yunus & Siti Nadiah Mohd Sapihie & Asm, 2023. "Model Predictive Control Based Energy Management System Literature Assessment for RES Integration," Energies, MDPI, vol. 16(8), pages 1-27, April.
    17. Ahmad Khan, Aftab & Naeem, Muhammad & Iqbal, Muhammad & Qaisar, Saad & Anpalagan, Alagan, 2016. "A compendium of optimization objectives, constraints, tools and algorithms for energy management in microgrids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 58(C), pages 1664-1683.
    18. Clarke, Will Challis & Brear, Michael John & Manzie, Chris, 2020. "Control of an isolated microgrid using hierarchical economic model predictive control," Applied Energy, Elsevier, vol. 280(C).
    19. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    20. Colmenar-Santos, Antonio & Linares-Mena, Ana-Rosa & Borge-Diez, David & Quinto-Alemany, Carlos-Domingo, 2017. "Impact assessment of electric vehicles on islands grids: A case study for Tenerife (Spain)," Energy, Elsevier, vol. 120(C), pages 385-396.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:11:y:2018:i:8:p:2010-:d:161526. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.