IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v263y2023ipcs036054422202744x.html
   My bibliography  Save this article

Endowing data-driven models with rejection ability: Out-of-distribution detection and confidence estimation for black-box models of building energy systems

Author

Listed:
  • Liang, Xinbin
  • Zhu, Xu
  • Chen, Kang
  • Chen, Siliang
  • Jin, Xinqiao
  • Du, Zhimin

Abstract

As one of the most popular modeling methods, data-driven methods would fail easily when the online data are outside the scope of training data. To tackle this problem, this paper proposed a novel concept named as the rejection ability of data-driven models. Its main idea is rejecting the unreliable predictions based on the threshold which can be either calculated from out-of-distribution detection model or the confidence score. To compare with traditional data-driven models, three evaluation metrics are derived from the rejection table to quantify model performance. The usefulness of rejection method is validated through the fault detection and diagnosis problem of chillers. The ASHARE RP-1043 dataset is adopted to conduct the data experiments where the in-distribution performance and out-of-distribution performance of three task models, including RF, ET and ANN, and their combinations with four rejection methods, including AE-MSE, AE-MAE, Entropy and Max probability, are comprehensively investigated. The experimental results indicate that the model performance improves for a large margin with the introduction of rejection method. The optimal combination is ET-Entropy model, where its threshold selection is discussed for different application scenarios. The proposed rejection methods might provide a solid foundation for the real-world application of data-driven models.

Suggested Citation

  • Liang, Xinbin & Zhu, Xu & Chen, Kang & Chen, Siliang & Jin, Xinqiao & Du, Zhimin, 2023. "Endowing data-driven models with rejection ability: Out-of-distribution detection and confidence estimation for black-box models of building energy systems," Energy, Elsevier, vol. 263(PC).
    • Handle: RePEc:eee:energy:v:263:y:2023:i:pc:s036054422202744x
    DOI: 10.1016/j.energy.2022.125858
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S036054422202744X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2022.125858?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhao, Yang & Wang, Shengwei & Xiao, Fu, 2013. "Pattern recognition-based chillers fault detection method using Support Vector Data Description (SVDD)," Applied Energy, Elsevier, vol. 112(C), pages 1041-1048.
    2. Zhang, Liang & Wen, Jin & Li, Yanfei & Chen, Jianli & Ye, Yunyang & Fu, Yangyang & Livingood, William, 2021. "A review of machine learning in building load prediction," Applied Energy, Elsevier, vol. 285(C).
    3. Zhu, Xu & Zhang, Shuai & Jin, Xinqiao & Du, Zhimin, 2020. "Deep learning based reference model for operational risk evaluation of screw chillers for energy efficiency," Energy, Elsevier, vol. 213(C).
    4. Bünning, Felix & Huber, Benjamin & Schalbetter, Adrian & Aboudonia, Ahmed & Hudoba de Badyn, Mathias & Heer, Philipp & Smith, Roy S. & Lygeros, John, 2022. "Physics-informed linear regression is competitive with two Machine Learning methods in residential building MPC," Applied Energy, Elsevier, vol. 310(C).
    5. Li, Ao & Xiao, Fu & Zhang, Chong & Fan, Cheng, 2021. "Attention-based interpretable neural network for building cooling load prediction," Applied Energy, Elsevier, vol. 299(C).
    6. Fan, Cheng & Xiao, Fu & Zhao, Yang & Wang, Jiayuan, 2018. "Analytical investigation of autoencoder-based methods for unsupervised anomaly detection in building energy data," Applied Energy, Elsevier, vol. 211(C), pages 1123-1135.
    7. Wang, Zhanwei & Wang, Zhiwei & He, Suowei & Gu, Xiaowei & Yan, Zeng Feng, 2017. "Fault detection and diagnosis of chillers using Bayesian network merged distance rejection and multi-source non-sensor information," Applied Energy, Elsevier, vol. 188(C), pages 200-214.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liang, Xinbin & Liu, Zhuoxuan & Wang, Jie & Jin, Xinqiao & Du, Zhimin, 2023. "Uncertainty quantification-based robust deep learning for building energy systems considering distribution shift problem," Applied Energy, Elsevier, vol. 337(C).
    2. Chen, Jianli & Zhang, Liang & Li, Yanfei & Shi, Yifu & Gao, Xinghua & Hu, Yuqing, 2022. "A review of computing-based automated fault detection and diagnosis of heating, ventilation and air conditioning systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 161(C).
    3. Wang, Zeyu & Liu, Jian & Zhang, Yuanxin & Yuan, Hongping & Zhang, Ruixue & Srinivasan, Ravi S., 2021. "Practical issues in implementing machine-learning models for building energy efficiency: Moving beyond obstacles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 143(C).
    4. Chen, Zhelun & O’Neill, Zheng & Wen, Jin & Pradhan, Ojas & Yang, Tao & Lu, Xing & Lin, Guanjing & Miyata, Shohei & Lee, Seungjae & Shen, Chou & Chiosa, Roberto & Piscitelli, Marco Savino & Capozzoli, , 2023. "A review of data-driven fault detection and diagnostics for building HVAC systems," Applied Energy, Elsevier, vol. 339(C).
    5. Liang, Xinbin & Zhu, Xu & Chen, Siliang & Jin, Xinqiao & Xiao, Fu & Du, Zhimin, 2023. "Physics-constrained cooperative learning-based reference models for smart management of chillers considering extrapolation scenarios," Applied Energy, Elsevier, vol. 349(C).
    6. Wang, Xinlin & Yao, Zhihao & Papaefthymiou, Marios, 2023. "A real-time electrical load forecasting and unsupervised anomaly detection framework," Applied Energy, Elsevier, vol. 330(PA).
    7. Zhu, Xu & Zhang, Shuai & Jin, Xinqiao & Du, Zhimin, 2020. "Deep learning based reference model for operational risk evaluation of screw chillers for energy efficiency," Energy, Elsevier, vol. 213(C).
    8. Tang, Lingfeng & Xie, Haipeng & Wang, Xiaoyang & Bie, Zhaohong, 2023. "Privacy-preserving knowledge sharing for few-shot building energy prediction: A federated learning approach," Applied Energy, Elsevier, vol. 337(C).
    9. Du, Zhimin & Liang, Xinbin & Chen, Siliang & Zhu, Xu & Chen, Kang & Jin, Xinqiao, 2023. "Knowledge-infused deep learning diagnosis model with self-assessment for smart management in HVAC systems," Energy, Elsevier, vol. 263(PD).
    10. Sun, Chunhua & Zhang, Haixiang & Cao, Shanshan & Xia, Guoqiang & Zhong, Jian & Wu, Xiangdong, 2023. "A hierarchical classifying and two-step training strategy for detection and diagnosis of anormal temperature in district heating system," Applied Energy, Elsevier, vol. 349(C).
    11. Li, Guannan & Li, Fan & Ahmad, Tanveer & Liu, Jiangyan & Li, Tao & Fang, Xi & Wu, Yubei, 2022. "Performance evaluation of sequence-to-sequence-Attention model for short-term multi-step ahead building energy predictions," Energy, Elsevier, vol. 259(C).
    12. Joe, Jaewan & Im, Piljae & Cui, Borui & Dong, Jin, 2023. "Model-based predictive control of multi-zone commercial building with a lumped building modelling approach," Energy, Elsevier, vol. 263(PA).
    13. Ghafoori, Mahdi & Abdallah, Moatassem & Kim, Serena, 2023. "Electricity peak shaving for commercial buildings using machine learning and vehicle to building (V2B) system," Applied Energy, Elsevier, vol. 340(C).
    14. Lei, Lei & Wu, Bing & Fang, Xin & Chen, Li & Wu, Hao & Liu, Wei, 2023. "A dynamic anomaly detection method of building energy consumption based on data mining technology," Energy, Elsevier, vol. 263(PA).
    15. Yong Zhou & Lingyu Wang & Junhao Qian, 2022. "Application of Combined Models Based on Empirical Mode Decomposition, Deep Learning, and Autoregressive Integrated Moving Average Model for Short-Term Heating Load Predictions," Sustainability, MDPI, vol. 14(12), pages 1-20, June.
    16. Zhao, Yang & Li, Tingting & Zhang, Xuejun & Zhang, Chaobo, 2019. "Artificial intelligence-based fault detection and diagnosis methods for building energy systems: Advantages, challenges and the future," Renewable and Sustainable Energy Reviews, Elsevier, vol. 109(C), pages 85-101.
    17. Li, Bingxu & Cheng, Fanyong & Zhang, Xin & Cui, Can & Cai, Wenjian, 2021. "A novel semi-supervised data-driven method for chiller fault diagnosis with unlabeled data," Applied Energy, Elsevier, vol. 285(C).
    18. Simona-Vasilica Oprea & Adela Bâra & Florina Camelia Puican & Ioan Cosmin Radu, 2021. "Anomaly Detection with Machine Learning Algorithms and Big Data in Electricity Consumption," Sustainability, MDPI, vol. 13(19), pages 1-20, October.
    19. Fan, Cheng & Sun, Yongjun & Zhao, Yang & Song, Mengjie & Wang, Jiayuan, 2019. "Deep learning-based feature engineering methods for improved building energy prediction," Applied Energy, Elsevier, vol. 240(C), pages 35-45.
    20. Xavier Serrano-Guerrero & Guillermo Escrivá-Escrivá & Santiago Luna-Romero & Jean-Michel Clairand, 2020. "A Time-Series Treatment Method to Obtain Electrical Consumption Patterns for Anomalies Detection Improvement in Electrical Consumption Profiles," Energies, MDPI, vol. 13(5), pages 1-23, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:263:y:2023:i:pc:s036054422202744x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.