Adaptive home energy management to self-motivated user preferences via iterative LLM-augmented reinforcement learning

Home energy management systems (HEMS) must balance competing objectives—electricity cost, thermal comfort, and carbon emissions—according to user preferences that are personalized, evolving, and often expressed in natural language. Conventional deep reinforcement learning (DRL) methods cannot directly interpret such preferences, while existing reinforcement learning (RL)+LLM integrations either invoke large language models (LLMs) at every control timestep or rely on one-shot preference parsing without feedback. This paper proposes LA-UPAHEM, an iterative LLM-augmented framework in which three specialized agents collaborate in a closed loop: a Code Generation Agent translates preferences into reward and state modification functions, a Result Analysis Agent diagnoses policy–preference misalignment from evaluation metrics, and an Optimal Performance Search Agent identifies the best-performing policy as the warm-start for the next iteration. LLMs are invoked only during this offline refinement phase; the deployed policy operates independently without LLM inference. Experiments on real residential energy datasets show that LA-UPAHEM outperforms classical DRL and existing RL+LLM baselines in both macro-level key performance indicator (KPI) alignment (cost, comfort, emissions) and micro-level rule compliance, achieving a Weighted Improvement Ratio (WIR) of 0.13 and a Rule Compliance Rate (RCR) of 0.84, while reducing the failure rate from 34.3% (static parsing) to 5.4%. The framework is robust to environmental noise, evaluation weight perturbation, and preference paraphrasing, and generalizes across multiple DRL algorithms, LLM backbones, and preference languages.