IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v415y2026ics0306261926006288.html

Adaptive home energy management to self-motivated user preferences via iterative LLM-augmented reinforcement learning

Author

Listed:
  • Du, Xiao
  • Luo, Fengji
  • Hu, Juntao
  • Zhou, Wei
  • Wen, Junhao

Abstract

Home energy management systems (HEMS) must balance competing objectives—electricity cost, thermal comfort, and carbon emissions—according to user preferences that are personalized, evolving, and often expressed in natural language. Conventional deep reinforcement learning (DRL) methods cannot directly interpret such preferences, while existing reinforcement learning (RL)+LLM integrations either invoke large language models (LLMs) at every control timestep or rely on one-shot preference parsing without feedback. This paper proposes LA-UPAHEM, an iterative LLM-augmented framework in which three specialized agents collaborate in a closed loop: a Code Generation Agent translates preferences into reward and state modification functions, a Result Analysis Agent diagnoses policy–preference misalignment from evaluation metrics, and an Optimal Performance Search Agent identifies the best-performing policy as the warm-start for the next iteration. LLMs are invoked only during this offline refinement phase; the deployed policy operates independently without LLM inference. Experiments on real residential energy datasets show that LA-UPAHEM outperforms classical DRL and existing RL+LLM baselines in both macro-level key performance indicator (KPI) alignment (cost, comfort, emissions) and micro-level rule compliance, achieving a Weighted Improvement Ratio (WIR) of 0.13 and a Rule Compliance Rate (RCR) of 0.84, while reducing the failure rate from 34.3% (static parsing) to 5.4%. The framework is robust to environmental noise, evaluation weight perturbation, and preference paraphrasing, and generalizes across multiple DRL algorithms, LLM backbones, and preference languages.

Suggested Citation

  • Du, Xiao & Luo, Fengji & Hu, Juntao & Zhou, Wei & Wen, Junhao, 2026. "Adaptive home energy management to self-motivated user preferences via iterative LLM-augmented reinforcement learning," Applied Energy, Elsevier, vol. 415(C).
  • Handle: RePEc:eee:appene:v:415:y:2026:i:c:s0306261926006288
    DOI: 10.1016/j.apenergy.2026.127976
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261926006288
    Download Restriction: Full text may be restricted for ScienceDirect subscribers

    File URL: https://libkey.io/10.1016/j.apenergy.2026.127976?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Liu, Yinyan & Ma, Jin & Xing, Xinjie & Liu, Xinglu & Wang, Wei, 2022. "A home energy management system incorporating data-driven uncertainty-aware user preference," Applied Energy, Elsevier, vol. 326(C).
    2. Tuomela, Sanna & de Castro Tomé, Mauricio & Iivari, Netta & Svento, Rauli, 2021. "Impacts of home energy management systems on electricity consumption," Applied Energy, Elsevier, vol. 299(C).
    3. Rokonuzzaman, Md. & Rahman, Saifur & Hannan, M.A. & Mishu, Mahmuda Khatun & Tan, Wen-Shan & Rahman, Kazi Sajedur & Pasupuleti, Jagadeesh & Amin, Nowshad, 2025. "Levenberg-Marquardt algorithm-based solar PV energy integrated internet of home energy management system," Applied Energy, Elsevier, vol. 378(PA).
    4. Ren, Kezheng & Liu, Jun & Wu, Zeyang & Liu, Xinglei & Nie, Yongxin & Xu, Haitao, 2024. "A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters," Applied Energy, Elsevier, vol. 355(C).
    5. Parrish, Bryony & Heptonstall, Phil & Gross, Rob & Sovacool, Benjamin K., 2020. "A systematic review of motivations, enablers and barriers for consumer engagement with residential demand response," Energy Policy, Elsevier, vol. 138(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Pengju & Chokwitthaya, Chanachok & Olofsson, Thomas & Lu, Weizhuo, 2026. "Demand response optimization incorporating thermal comfort in single-family houses with on-site generation: a systematic review," Applied Energy, Elsevier, vol. 406(C).
    2. Zhang, Xunning & Cheng, Yuheng & Gui, Xuanang & Zhao, Huan & Zhao, Junhua & Yan, Jinyue, 2026. "Large language model-enhanced home energy management with dynamic user preference elicitation and hierarchical data-sharing," Applied Energy, Elsevier, vol. 410(C).
    3. Liu, Weifeng & Shen, Yu & Tian, Jiyuan & Meng, Yuhang & Wu, Qing & He, Guangyu, 2025. "Enabling self-approaching optimization of Home Energy Management System through multi-agent systems," Energy, Elsevier, vol. 334(C).
    4. Rokonuzzaman, Md. & Rahman, Saifur & Hannan, M.A. & Mishu, Mahmuda Khatun & Tan, Wen-Shan & Rahman, Kazi Sajedur & Pasupuleti, Jagadeesh & Amin, Nowshad, 2025. "Levenberg-Marquardt algorithm-based solar PV energy integrated internet of home energy management system," Applied Energy, Elsevier, vol. 378(PA).
    5. Huang, Zhijia & Wang, Fang & Lu, Yuehong & Chen, Xiaofeng & Wu, Qiqi, 2023. "Optimization model for home energy management system of rural dwellings," Energy, Elsevier, vol. 283(C).
    6. Lehmann, Nico & Sloot, Daniel & Schüle, Christopher & Ardone, Armin & Fichtner, Wolf, 2023. "The motivational drivers behind consumer preferences for regional electricity – Results of a choice experiment in Southern Germany," Energy Economics, Elsevier, vol. 120(C).
    7. Chen, Ting & Vandendriessche, Frederik, 2023. "Enabling independent flexibility service providers to participate in electricity markets: A legal analysis of the Belgium case," Utilities Policy, Elsevier, vol. 81(C).
    8. Anna Borawska & Mariusz Borawski & Małgorzata Łatuszyńska, 2022. "Effectiveness of Electricity-Saving Communication Campaigns: Neurophysiological Approach," Energies, MDPI, vol. 15(4), pages 1-19, February.
    9. Sæle, Hanne & Sperstad, Iver Bakken & Wang Hoiem, Kristian & Mathiesen, Vivi, 2023. "Understanding barriers to utilising flexibility in operation and planning of the electricity distribution system – Classification frameworks with applications to Norway," Energy Policy, Elsevier, vol. 180(C).
    10. Olga Bogdanova & Karīna Viskuba & Laila Zemīte, 2023. "A Review of Barriers and Enables in Demand Response Performance Chain," Energies, MDPI, vol. 16(18), pages 1-33, September.
    11. Pla, Benjamín & Bares, Pau & Aronis, André Nakaema & Anuratha, Sanjith, 2024. "Leveraging battery electric vehicle energy storage potential for home energy saving by model predictive control with backward induction," Applied Energy, Elsevier, vol. 372(C).
    12. Ioanna-M. Chatzigeorgiou & Christos Diou & Kyriakos C. Chatzidimitriou & Georgios T. Andreou, 2021. "Demand Response Alert Service Based on Appliance Modeling," Energies, MDPI, vol. 14(10), pages 1-15, May.
    13. Wang, Aijia & Mah, Daphne Ngar-yin, 2026. "A systematic review on varieties, effectiveness and determinants of consumer engagement in the context of electricity market reforms from a European-Asian comparative perspective," Renewable and Sustainable Energy Reviews, Elsevier, vol. 226(PD).
    14. Murat Tasci & Hidir Duzkaya, 2025. "Estimation of Working Error of Electricity Meter Using Artificial Neural Network (ANN)," Energies, MDPI, vol. 18(5), pages 1-16, March.
    15. Osunmuyiwa, Olufolahan O. & Peacock, Andrew D. & Payne, Sarah R. & Vigneswara Ilavarasan, P. & Jenkins, David P., 2021. "Divergent imaginaries? Co-producing practitioner and householder perspective to cooling demand response in India," Energy Policy, Elsevier, vol. 152(C).
    16. Ruben Barreto & Calvin Gonçalves & Luis Gomes & Pedro Faria & Zita Vale, 2022. "Evaluation Metrics to Assess the Most Suitable Energy Community End-Users to Participate in Demand Response," Energies, MDPI, vol. 15(7), pages 1-18, March.
    17. Sarran, Lucile & Gunay, H. Burak & O'Brien, William & Hviid, Christian A. & Rode, Carsten, 2021. "A data-driven study of thermostat overrides during demand response events," Energy Policy, Elsevier, vol. 153(C).
    18. Jiajun Ning & Xiyu Li & Yuan Gao, 2025. "Dynamic analysis and influence mechanism of digital technology diffusion in the energy industry based on the evolutionary game model of complex networks," Energy & Environment, , vol. 36(7), pages 3355-3375, November.
    19. Lin, Jin & Dong, Jun & Liu, Dongran & Zhang, Yaoyu & Ma, Tongtao, 2022. "From peak shedding to low-carbon transitions: Customer psychological factors in demand response," Energy, Elsevier, vol. 238(PA).
    20. Guo, Xiaochao & Wang, Chao & Wu, Tao & Li, Ruiheng & Zhu, Houyi & Zhang, Huaiqing, 2023. "Detecting the novel appliance in non-intrusive load monitoring," Applied Energy, Elsevier, vol. 343(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:415:y:2026:i:c:s0306261926006288. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.