IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v156y2015icp577-586.html
   My bibliography  Save this article

Reinforcement learning for optimal control of low exergy buildings

Author

Listed:
  • Yang, Lei
  • Nagy, Zoltan
  • Goffin, Philippe
  • Schlueter, Arno

Abstract

Over a third of the anthropogenic greenhouse gas (GHG) emissions stem from cooling and heating buildings, due to their fossil fuel based operation. Low exergy building systems are a promising approach to reduce energy consumption as well as GHG emissions. They consists of renewable energy technologies, such as PV, PV/T and heat pumps. Since careful tuning of parameters is required, a manual setup may result in sub-optimal operation. A model predictive control approach is unnecessarily complex due to the required model identification. Therefore, in this work we present a reinforcement learning control (RLC) approach. The studied building consists of a PV/T array for solar heat and electricity generation, as well as geothermal heat pumps. We present RLC for the PV/T array, and the full building model. Two methods, Tabular Q-learning and Batch Q-learning with Memory Replay, are implemented with real building settings and actual weather conditions in a Matlab/Simulink framework. The performance is evaluated against standard rule-based control (RBC). We investigated different neural network structures and find that some outperformed RBC already during the learning phase. Overall, every RLC strategy for PV/T outperformed RBC by over 10% after the third year. Likewise, for the full building, RLC outperforms RBC in terms of meeting the heating demand, maintaining the optimal operation temperature and compensating more effectively for ground heat. This allows to reduce engineering costs associated with the setup of these systems, as well as decrease the return-of-invest period, both of which are necessary to create a sustainable, zero-emission building stock.

Suggested Citation

  • Yang, Lei & Nagy, Zoltan & Goffin, Philippe & Schlueter, Arno, 2015. "Reinforcement learning for optimal control of low exergy buildings," Applied Energy, Elsevier, vol. 156(C), pages 577-586.
    • Handle: RePEc:eee:appene:v:156:y:2015:i:c:p:577-586
    DOI: 10.1016/j.apenergy.2015.07.050
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030626191500879X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2015.07.050?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Široký, Jan & Oldewurtel, Frauke & Cigler, Jiří & Prívara, Samuel, 2011. "Experimental analysis of model predictive control for an energy efficient building heating system," Applied Energy, Elsevier, vol. 88(9), pages 3079-3087.
    2. Ben-Nakhi, Abdullatif E. & Mahmoud, Mohamed A., 2002. "Energy conservation in buildings through efficient A/C control using neural networks," Applied Energy, Elsevier, vol. 73(1), pages 5-23, September.
    3. Petersen, Steffen & Bundgaard, Katrine Wieck, 2014. "The effect of weather forecast uncertainty on a predictive control concept for building systems operation," Applied Energy, Elsevier, vol. 116(C), pages 311-321.
    4. Oldewurtel, Frauke & Sturzenegger, David & Morari, Manfred, 2013. "Importance of occupancy information for building climate control," Applied Energy, Elsevier, vol. 101(C), pages 521-532.
    5. Goyal, Siddharth & Ingley, Herbert A. & Barooah, Prabir, 2013. "Occupancy-based zone-climate control for energy-efficient buildings: Complexity vs. performance," Applied Energy, Elsevier, vol. 106(C), pages 209-221.
    6. Meggers, Forrest & Ritter, Volker & Goffin, Philippe & Baetschmann, Marc & Leibundgut, Hansjürg, 2012. "Low exergy building systems implementation," Energy, Elsevier, vol. 41(1), pages 48-55.
    7. Kalogirou, Soteris A., 2000. "Applications of artificial neural-networks for energy systems," Applied Energy, Elsevier, vol. 67(1-2), pages 17-35, September.
    8. Dounis, A.I. & Caraiscos, C., 2009. "Advanced control systems engineering for energy and comfort management in a building environment--A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 13(6-7), pages 1246-1261, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gianluca Serale & Massimo Fiorentini & Alfonso Capozzoli & Daniele Bernardini & Alberto Bemporad, 2018. "Model Predictive Control (MPC) for Enhancing Building and HVAC System Energy Efficiency: Problem Formulation, Applications and Opportunities," Energies, MDPI, vol. 11(3), pages 1-35, March.
    2. Korkas, Christos D. & Baldi, Simone & Michailidis, Iakovos & Kosmatopoulos, Elias B., 2015. "Intelligent energy and thermal comfort management in grid-connected microgrids with heterogeneous occupancy schedule," Applied Energy, Elsevier, vol. 149(C), pages 194-203.
    3. Žáčeková, Eva & Váňa, Zdeněk & Cigler, Jiří, 2014. "Towards the real-life implementation of MPC for an office building: Identification issues," Applied Energy, Elsevier, vol. 135(C), pages 53-62.
    4. Gruber, Mattias & Trüschel, Anders & Dalenbäck, Jan-Olof, 2015. "Energy efficient climate control in office buildings without giving up implementability," Applied Energy, Elsevier, vol. 154(C), pages 934-943.
    5. Ahn, Jonghoon & Cho, Soolyeon & Chung, Dae Hun, 2017. "Analysis of energy and control efficiencies of fuzzy logic and artificial neural network technologies in the heating energy supply system responding to the changes of user demands," Applied Energy, Elsevier, vol. 190(C), pages 222-231.
    6. Paulína Šujanová & Monika Rychtáriková & Tiago Sotto Mayor & Affan Hyder, 2019. "A Healthy, Energy-Efficient and Comfortable Indoor Environment, a Review," Energies, MDPI, vol. 12(8), pages 1-37, April.
    7. Yang, Zheng & Becerik-Gerber, Burcin, 2015. "A model calibration framework for simultaneous multi-level building energy simulation," Applied Energy, Elsevier, vol. 149(C), pages 415-431.
    8. Yang, Liu & Yan, Haiyan & Lam, Joseph C., 2014. "Thermal comfort and building energy consumption implications – A review," Applied Energy, Elsevier, vol. 115(C), pages 164-173.
    9. Nägele, Florian & Kasper, Thomas & Girod, Bastien, 2017. "Turning up the heat on obsolete thermostats: A simulation-based comparison of intelligent control approaches for residential heating systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 75(C), pages 1254-1268.
    10. Dong, Bing & Li, Zhaoxuan & Taha, Ahmad & Gatsis, Nikolaos, 2018. "Occupancy-based buildings-to-grid integration framework for smart and connected communities," Applied Energy, Elsevier, vol. 219(C), pages 123-137.
    11. Barbeito, Inés & Zaragoza, Sonia & Tarrío-Saavedra, Javier & Naya, Salvador, 2017. "Assessing thermal comfort and energy efficiency in buildings by statistical quality control for autocorrelated data," Applied Energy, Elsevier, vol. 190(C), pages 1-17.
    12. Barbara Mayer & Michaela Killian & Martin Kozek, 2017. "Hierarchical Model Predictive Control for Sustainable Building Automation," Sustainability, MDPI, vol. 9(2), pages 1-20, February.
    13. Muhammad Fayaz & DoHyeun Kim, 2018. "Energy Consumption Optimization and User Comfort Management in Residential Buildings Using a Bat Algorithm and Fuzzy Logic," Energies, MDPI, vol. 11(1), pages 1-22, January.
    14. Bhowmik, Chiranjib & Bhowmik, Sumit & Ray, Amitava & Pandey, Krishna Murari, 2017. "Optimal green energy planning for sustainable development: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 71(C), pages 796-813.
    15. Hou, Juan & Li, Haoran & Nord, Natasa, 2022. "Nonlinear model predictive control for the space heating system of a university building in Norway," Energy, Elsevier, vol. 253(C).
    16. Jiang, Lai & Yao, Runming & Liu, Kecheng & McCrindle, Rachel, 2017. "An Epistemic-Deontic-Axiologic (EDA) agent-based energy management system in office buildings," Applied Energy, Elsevier, vol. 205(C), pages 440-452.
    17. Enescu, Diana, 2017. "A review of thermal comfort models and indicators for indoor environments," Renewable and Sustainable Energy Reviews, Elsevier, vol. 79(C), pages 1353-1379.
    18. Wang, Zhu & Wang, Lingfeng & Dounis, Anastasios I. & Yang, Rui, 2012. "Multi-agent control system with information fusion based comfort model for smart buildings," Applied Energy, Elsevier, vol. 99(C), pages 247-254.
    19. Azar, Elie & Nikolopoulou, Christina & Papadopoulos, Sokratis, 2016. "Integrating and optimizing metrics of sustainable building performance using human-focused agent-based modeling," Applied Energy, Elsevier, vol. 183(C), pages 926-937.
    20. Byung-Ki Jeon & Eui-Jong Kim, 2021. "LSTM-Based Model Predictive Control for Optimal Temperature Set-Point Planning," Sustainability, MDPI, vol. 13(2), pages 1-14, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:156:y:2015:i:c:p:577-586. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.