IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i24p8210-d697081.html
   My bibliography  Save this article

Energy Management Model for HVAC Control Supported by Reinforcement Learning

Author

Listed:
  • Pedro Macieira

    (GECAD—Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO), P-4200-072 Porto, Portugal)

  • Luis Gomes

    (GECAD—Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO), P-4200-072 Porto, Portugal)

  • Zita Vale

    (GECAD—Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO), P-4200-072 Porto, Portugal)

Abstract

Heating, ventilating, and air conditioning (HVAC) units account for a significant consumption share in buildings, namely office buildings. Therefore, this paper addresses the possibility of having an intelligent and more cost-effective solution for the management of HVAC units in office buildings. The method applied in this paper divides the addressed problem into three steps: (i) the continuous acquisition of data provided by an open-source building energy management systems, (ii) the proposed learning and predictive model able to predict if users will be working in a given location, and (iii) the proposed decision model to manage the HVAC units according to the prediction of users, current environmental context, and current energy prices. The results show that the proposed predictive model was able to achieve a 93.8% accuracy and that the proposed decision tree enabled the maintenance of users’ comfort. The results demonstrate that the proposed solution is able to run in real-time in a real office building, making it a possible solution for smart buildings.

Suggested Citation

  • Pedro Macieira & Luis Gomes & Zita Vale, 2021. "Energy Management Model for HVAC Control Supported by Reinforcement Learning," Energies, MDPI, vol. 14(24), pages 1-14, December.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:24:p:8210-:d:697081
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/24/8210/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/24/8210/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dana-Mihaela Petroșanu & George Căruțașu & Nicoleta Luminița Căruțașu & Alexandru Pîrjan, 2019. "A Review of the Recent Developments in Integrating Machine Learning Models with Sensor Devices in the Smart Buildings Sector with a View to Attaining Enhanced Sensing, Energy Efficiency, and Optimal B," Energies, MDPI, vol. 12(24), pages 1-64, December.
    2. Jung, Wooyoung & Jazizadeh, Farrokh, 2019. "Human-in-the-loop HVAC operations: A quantitative review on occupancy, comfort, and energy-efficiency dimensions," Applied Energy, Elsevier, vol. 239(C), pages 1471-1508.
    3. Ingo Liere-Netheler & Frank Schuldt & Karsten von Maydell & Carsten Agert, 2020. "Simulation of Incidental Distributed Generation Curtailment to Maximize the Integration of Renewable Energy Generation in Power Systems," Energies, MDPI, vol. 13(16), pages 1-22, August.
    4. Robert Lou & Kevin P. Hallinan & Kefan Huang & Timothy Reissman, 2020. "Smart Wifi Thermostat-Enabled Thermal Comfort Control in Residences," Sustainability, MDPI, vol. 12(5), pages 1-15, March.
    5. Adhikari, Rajendra & Pipattanasomporn, M. & Rahman, S., 2018. "An algorithm for optimal management of aggregated HVAC power demand using smart thermostats," Applied Energy, Elsevier, vol. 217(C), pages 166-177.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    2. Omar al-Ani & Sanjoy Das & Hongyu Wu, 2023. "Imitation Learning with Deep Attentive Tabular Neural Networks for Environmental Prediction and Control in Smart Home," Energies, MDPI, vol. 16(13), pages 1-19, June.
    3. Guillermo Bejarano & João M. Lemos & Javier Rico-Azagra & Francisco R. Rubio & Manuel G. Ortega, 2022. "Energy Management of Refrigeration Systems with Thermal Energy Storage Based on Non-Linear Model Predictive Control," Mathematics, MDPI, vol. 10(17), pages 1-27, September.
    4. Prasertsak Charoen & Nathavuth Kitbutrawat & Jasada Kudtongngam, 2022. "A Demand Response Implementation with Building Energy Management System," Energies, MDPI, vol. 15(3), pages 1-21, February.
    5. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    6. Dongsu Kim & Jongman Lee & Sunglok Do & Pedro J. Mago & Kwang Ho Lee & Heejin Cho, 2022. "Energy Modeling and Model Predictive Control for HVAC in Buildings: A Review of Current Research Trends," Energies, MDPI, vol. 15(19), pages 1-30, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefano Villa & Claudio Sassanelli, 2020. "The Data-Driven Multi-Step Approach for Dynamic Estimation of Buildings’ Interior Temperature," Energies, MDPI, vol. 13(24), pages 1-23, December.
    2. Tarragona, Joan & Pisello, Anna Laura & Fernández, Cèsar & Cabeza, Luisa F. & Payá, Jorge & Marchante-Avellaneda, Javier & de Gracia, Alvaro, 2022. "Analysis of thermal energy storage tanks and PV panels combinations in different buildings controlled through model predictive control," Energy, Elsevier, vol. 239(PC).
    3. Marek Borowski & Klaudia Zwolińska & Marcin Czerwiński, 2022. "An Experimental Study of Thermal Comfort and Indoor Air Quality—A Case Study of a Hotel Building," Energies, MDPI, vol. 15(6), pages 1-18, March.
    4. Dong, Zihang & Zhang, Xi & Li, Yijun & Strbac, Goran, 2023. "Values of coordinated residential space heating in demand response provision," Applied Energy, Elsevier, vol. 330(PB).
    5. Nilofar Asim & Marzieh Badiei & Masita Mohammad & Halim Razali & Armin Rajabi & Lim Chin Haw & Mariyam Jameelah Ghazali, 2022. "Sustainability of Heating, Ventilation and Air-Conditioning (HVAC) Systems in Buildings—An Overview," IJERPH, MDPI, vol. 19(2), pages 1-16, January.
    6. Filipe Soares & André Madureira & Andreu Pagès & António Barbosa & António Coelho & Fernando Cassola & Fernando Ribeiro & João Viana & José Andrade & Marina Dorokhova & Nélson Morais & Nicolas Wyrsch , 2021. "FEEdBACk: An ICT-Based Platform to Increase Energy Efficiency through Buildings’ Consumer Engagement," Energies, MDPI, vol. 14(6), pages 1-43, March.
    7. Yuri Bulatov & Andrey Kryukov & Andrey Batuhtin & Konstantin Suslov & Ksenia Korotkova & Denis Sidorov, 2022. "Digital Twin Formation Method for Distributed Generation Plants of Cyber–Physical Power Supply Systems," Mathematics, MDPI, vol. 10(16), pages 1-19, August.
    8. Sarah O’Connell & Marcus Martin Keane, 2021. "Development of a Framework for Activation of Aggregator Led Flexibility," Energies, MDPI, vol. 14(16), pages 1-15, August.
    9. Nizami, Sohrab & Tushar, Wayes & Hossain, M.J. & Yuen, Chau & Saha, Tapan & Poor, H. Vincent, 2022. "Transactive energy for low voltage residential networks: A review," Applied Energy, Elsevier, vol. 323(C).
    10. Max Bodenstein & Ingo Liere-Netheler & Frank Schuldt & Karsten von Maydell & Alexander K. Hartmann & Carsten Agert, 2023. "Optimized Power Flow Control to Minimize Congestion in a Modern Power System," Energies, MDPI, vol. 16(12), pages 1-19, June.
    11. Halhoul Merabet, Ghezlane & Essaaidi, Mohamed & Ben Haddou, Mohamed & Qolomany, Basheer & Qadir, Junaid & Anan, Muhammad & Al-Fuqaha, Ala & Abid, Mohamed Riduan & Benhaddou, Driss, 2021. "Intelligent building control systems for thermal comfort and energy-efficiency: A systematic review of artificial intelligence-assisted techniques," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    12. Abdulelah D. Alhamayani & Qiancheng Sun & Kevin P. Hallinan, 2021. "Estimating Smart Wi-Fi Thermostat-Enabled Thermal Comfort Control Savings for Any Residence," Clean Technol., MDPI, vol. 3(4), pages 1-18, October.
    13. Calise, F. & Cappiello, F. & D'Agostino, D. & Vicidomini, M., 2021. "Heat metering for residential buildings: A novel approach through dynamic simulations for the calculation of energy and economic savings," Energy, Elsevier, vol. 234(C).
    14. Gao, Zhikun & Yu, Junqi & Zhao, Anjun & Hu, Qun & Yang, Siyuan, 2022. "A hybrid method of cooling load forecasting for large commercial building based on extreme learning machine," Energy, Elsevier, vol. 238(PC).
    15. Wei, Shuangyu & Tien, Paige Wenbin & Calautit, John Kaiser & Wu, Yupeng & Boukhanouf, Rabah, 2020. "Vision-based detection and prediction of equipment heat gains in commercial office buildings using a deep learning method," Applied Energy, Elsevier, vol. 277(C).
    16. Su, Bing & Wang, Shengwei, 2020. "An agent-based distributed real-time optimal control strategy for building HVAC systems for applications in the context of future IoT-based smart sensor networks," Applied Energy, Elsevier, vol. 274(C).
    17. Muhammad Saidu Aliero & Muhammad Asif & Imran Ghani & Muhammad Fermi Pasha & Seung Ryul Jeong, 2022. "Systematic Review Analysis on Smart Building: Challenges and Opportunities," Sustainability, MDPI, vol. 14(5), pages 1-28, March.
    18. Barone, G. & Buonomano, A. & Forzano, C. & Giuzio, G.F. & Palombo, A. & Russo, G., 2023. "A new thermal comfort model based on physiological parameters for the smart design and control of energy-efficient HVAC systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).
    19. Zeng, Yuan & Zhang, Ruiwen & Wang, Dong & Mu, Yunfei & Jia, Hongjie, 2019. "A regional power grid operation and planning method considering renewable energy generation and load control," Applied Energy, Elsevier, vol. 237(C), pages 304-313.
    20. Greve, Thomas & Teng, Fei & Pollitt, Michael G. & Strbac, Goran, 2018. "A system operator’s utility function for the frequency response market," Applied Energy, Elsevier, vol. 231(C), pages 562-569.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:24:p:8210-:d:697081. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.