IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v401y2025ipas030626192501400x.html

A systematic review of transformers and large language models in the energy sector: towards agentic digital twins

Author

Listed:
  • Antonesi, Gabriel
  • Cioara, Tudor
  • Anghel, Ionut
  • Michalakopoulos, Vasilis
  • Sarmas, Elissaios
  • Toderean, Liana

Abstract

Artificial intelligence (AI) has long promised to improve energy management in smart grids by enhancing situational awareness and supporting more effective decision-making. While traditional machine learning has demonstrated notable results in forecasting and optimization, it often struggles with generalization, situational awareness, and heterogeneous data integration. Recent advances in Transformer architecture, including Large Language Models (LLMs) and Foundation Models (FMs) can significantly improve the ability to model complex temporal and contextual relationships, as well as in multi-modal data fusion which is valuable for most AI applications in the energy sector. In this review, we synthesize the rapidly expanding field of AI applications in the energy domain, with a focus on Transformer Models (TMs) and LLMs, which have shown growing relevance. We examine the architectural foundations, domain-specific adaptations and practical implementations of TMs across various forecasting and grid management tasks. We then explore the emerging role of LLMs in the field: adaptation and fine tuning for the energy sector, the type of tasks they are suited for, and the new challenges they introduce. Along the way, we highlight practical implementations, innovations, and areas where the research frontier is rapidly expanding. These recent developments reviewed underscore a broader trend: Generative AI (GenAI) is beginning to augment decision-making not only in high-level planning but also in day-to-day operations, from forecasting and grid balancing to workforce training and asset onboarding. While FMs hold promises, we found limited evidence of their concrete application in energy domain to date. Therefore we introduce the concept of the Agentic Digital Twin, a next-generation model that integrates FMs to bring multi-modal situational awareness, autonomy, proactivity, and social interaction into digital twin-based energy management systems. We present the transformational impact of FMs to each phase of a digital twin and identify the open challenges that need to be addressed for their efficient and effective integration.

Suggested Citation

  • Antonesi, Gabriel & Cioara, Tudor & Anghel, Ionut & Michalakopoulos, Vasilis & Sarmas, Elissaios & Toderean, Liana, 2025. "A systematic review of transformers and large language models in the energy sector: towards agentic digital twins," Applied Energy, Elsevier, vol. 401(PA).
    • Handle: RePEc:eee:appene:v:401:y:2025:i:pa:s030626192501400x
    DOI: 10.1016/j.apenergy.2025.126670
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030626192501400X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2025.126670?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Dong, Changyin & Xiong, Zhuozhi & Zhang, Chu & Li, Ni & Li, Ye & Xie, Ning & Zhang, Jiarui & Wang, Hao, 2025. "A transformer-based approach for deep feature extraction and energy consumption prediction of electric buses based on driving distances," Applied Energy, Elsevier, vol. 380(C).
    2. Bao, Gengyi & Liu, Xinhua & Zou, Bosong & Yang, Kaiyi & Zhao, Junwei & Zhang, Lisheng & Chen, Muyang & Qiao, Yuanting & Wang, Wentao & Tan, Rui & Wang, Xiangwen, 2025. "Collaborative framework of Transformer and LSTM for enhanced state-of-charge estimation in lithium-ion batteries," Energy, Elsevier, vol. 322(C).
    3. Zhang, Zongbin & Huang, Xiaoqiao & Li, Chengli & Cheng, Feiyan & Tai, Yonghang, 2025. "CRAformer: A cross-residual attention transformer for solar irradiation multistep forecasting," Energy, Elsevier, vol. 320(C).
    4. Wang, Chao-fan & Liu, Kui-xing & Peng, Jieyang & Li, Xiang & Liu, Xiu-feng & Zhang, Jia-wan & Niu, Zhi-bin, 2025. "High-precision energy consumption forecasting for large office building using a signal decomposition-based deep learning approach," Energy, Elsevier, vol. 314(C).
    5. Wu, Tangjie & Ling, Qiang, 2024. "STELLM: Spatio-temporal enhanced pre-trained large language model for wind speed forecasting," Applied Energy, Elsevier, vol. 375(C).
    6. Chen Gao & Xiaochong Lan & Nian Li & Yuan Yuan & Jingtao Ding & Zhilun Zhou & Fengli Xu & Yong Li, 2024. "Large language models empowered agent-based modeling and simulation: a survey and perspectives," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 11(1), pages 1-24, December.
    7. Jiang, Canchen & Liebman, Ariel & Jie, Bo & Wang, Hao, 2025. "Dynamic rolling horizon optimization for network-constrained V2X value stacking of electric vehicles under uncertainties," Renewable Energy, Elsevier, vol. 244(C).
    8. Dong, Hanjiang & Zhu, Jizhong & Li, Shenglin & Wu, Wanli & Zhu, Haohao & Fan, Junwei, 2023. "Short-term residential household reactive power forecasting considering active power demand via deep Transformer sequence-to-sequence networks," Applied Energy, Elsevier, vol. 329(C).
    9. Lin, Zijie & Xie, Linbo & Zhang, Siyuan, 2024. "A compound framework for short-term gas load forecasting combining time-enhanced perception transformer and two-stage feature extraction," Energy, Elsevier, vol. 298(C).
    10. Zhu, Leyang & Huang, Xiaoqiao & Zhang, Zongbin & Li, Chengli & Tai, Yonghang, 2025. "A novel U-LSTM-AFT model for hourly solar irradiance forecasting," Renewable Energy, Elsevier, vol. 238(C).
    11. Ma, Yichuan X. & Yeung, Lawrence K., 2024. "BEForeGAN: An image-based deep generative approach for day-ahead forecasting of building HVAC energy consumption," Applied Energy, Elsevier, vol. 376(PA).
    12. Wu, Qiang & Zheng, Hongling & Guo, Xiaozhu & Liu, Guangqiang, 2022. "Promoting wind energy for sustainable development by precise wind speed prediction based on graph neural networks," Renewable Energy, Elsevier, vol. 199(C), pages 977-992.
    13. Nascimento, Erick Giovani Sperandio & de Melo, Talison A.C. & Moreira, Davidson M., 2023. "A transformer-based deep neural network with wavelet transform for forecasting wind speed and wind energy," Energy, Elsevier, vol. 278(C).
    14. Feng, Zhengyuan & Sun, Yuheng & Ning, Jun & Tang, Shoujuan & Liu, Guangxin & Liu, Fangtao & Li, Yang & Shi, Lei, 2025. "Implementing a provincial-level universal daily industrial carbon emissions prediction by fine-tuning the large language model," Applied Energy, Elsevier, vol. 383(C).
    15. Liao, Wenlong & Wang, Shouxiang & Yang, Dechang & Yang, Zhe & Fang, Jiannong & Rehtanz, Christian & Porté-Agel, Fernando, 2025. "TimeGPT in load forecasting: A large time series model perspective," Applied Energy, Elsevier, vol. 379(C).
    16. Liao, Xuan & Wong, Man Sing & Zhu, Rui, 2025. "Dual-gate Temporal Fusion Transformer for estimating large-scale land surface solar irradiation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 214(C).
    17. Liang, Xuefeng & Hu, Zetian & Zhang, Jun & Chen, Han & Gu, Qingshui & You, Xiaochuan, 2025. "Developing a robust wind power forecasting method: Integrating data repair, feature screening, and economic impact analysis for practical applications," Renewable Energy, Elsevier, vol. 247(C).
    18. Kai Li & Jason Rollins & Erjia Yan, 2018. "Web of Science use in published research and review papers 1997–2017: a selective, dynamic, cross-domain, content-based analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 1-20, April.
    19. Wang, Yun & Xu, Houhua & Song, Mengmeng & Zhang, Fan & Li, Yifen & Zhou, Shengchao & Zhang, Lingjun, 2023. "A convolutional Transformer-based truncated Gaussian density network with data denoising for wind speed forecasting," Applied Energy, Elsevier, vol. 333(C).
    20. Zhang, Liwenbo & Wilson, Robin & Sumner, Mark & Wu, Yupeng, 2023. "Advanced multimodal fusion method for very short-term solar irradiance forecasting using sky images and meteorological data: A gate and transformer mechanism approach," Renewable Energy, Elsevier, vol. 216(C).
    21. Semmelmann, Leo & Hertel, Matthias & Kircher, Kevin J. & Mikut, Ralf & Hagenmeyer, Veit & Weinhardt, Christof, 2024. "The impact of heat pumps on day-ahead energy community load forecasting," Applied Energy, Elsevier, vol. 368(C).
    22. Monaco, Roberto & Bergaentzlé, Claire & Leiva Vilaplana, Jose Angel & Ackom, Emmanuel & Nielsen, Per Sieverts, 2024. "Digitalization of power distribution grids: Barrier analysis, ranking and policy recommendations," Energy Policy, Elsevier, vol. 188(C).
    23. Bashir, Tasarruf & Wang, Huifang & Tahir, Mustafa & Zhang, Yixiang, 2025. "Wind and solar power forecasting based on hybrid CNN-ABiLSTM, CNN-transformer-MLP models," Renewable Energy, Elsevier, vol. 239(C).
    24. Matthew J Page & Joanne E McKenzie & Patrick M Bossuyt & Isabelle Boutron & Tammy C Hoffmann & Cynthia D Mulrow & Larissa Shamseer & Jennifer M Tetzlaff & Elie A Akl & Sue E Brennan & Roger Chou & Jul, 2021. "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," PLOS Medicine, Public Library of Science, vol. 18(3), pages 1-15, March.
    25. Liu, Jingxuan & Zang, Haixiang & Cheng, Lilin & Ding, Tao & Wei, Zhinong & Sun, Guoqiang, 2023. "A Transformer-based multimodal-learning framework using sky images for ultra-short-term solar irradiance forecasting," Applied Energy, Elsevier, vol. 342(C).
    26. Zhang, Chaobo & Zhang, Jian & Zhao, Yang & Lu, Jie, 2025. "Automated data-driven building energy load prediction method based on generative pre-trained transformers (GPT)," Energy, Elsevier, vol. 318(C).
    27. Jiang, Gang & Ma, Zhihao & Zhang, Liang & Chen, Jianli, 2024. "EPlus-LLM: A large language model-based computing platform for automated building energy modeling," Applied Energy, Elsevier, vol. 367(C).
    28. Zhang, Liang & Chen, Zhelun, 2025. "Opportunities of applying Large Language Models in building energy sector," Renewable and Sustainable Energy Reviews, Elsevier, vol. 214(C).
    29. Liu, Jiarui & Fu, Yuchen, 2023. "Renewable energy forecasting: A self-supervised learning-based transformer variant," Energy, Elsevier, vol. 284(C).
    30. Zheng, Peijun & Zhou, Heng & Liu, Jiang & Nakanishi, Yosuke, 2023. "Interpretable building energy consumption forecasting using spectral clustering algorithm and temporal fusion transformers architecture," Applied Energy, Elsevier, vol. 349(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lee, Jehyung & Jeong, Gil & Choi, Hyungwon & Kim, Hyunjong & Cho, Eunjun & Seo, Beomsoo & Chin, Simon & Kang, Yong Tae, 2025. "Oil circulation rate prediction by transformer based deep learning model in R290 heat pump systems," Energy, Elsevier, vol. 337(C).
    2. Hui, Tan Kah & Hwee, Ah Khoon & QingHua, Liu & Jing, Khoo Terh & Ohueri, Chukwuka Christian, 2026. "From state-of-the-art to next-generation solutions: A review for Gen-AI integration in photovoltaic maintenance," Applied Energy, Elsevier, vol. 402(PB).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hussan, Umair & Wang, Huaizhi & Peng, Jianchun & Jiang, Hui & Rasheed, Hamna, 2026. "Transformer-based renewable energy forecasting: A comprehensive review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 226(PC).
    2. Verdone, Alessio & Panella, Massimo & De Santis, Enrico & Rizzi, Antonello, 2025. "A review of solar and wind energy forecasting: From single-site to multi-site paradigm," Applied Energy, Elsevier, vol. 392(C).
    3. Qu, Kai & Xue, Shuangsi & Zheng, Xiaodong & Yan, Dapeng & Cao, Hui, 2026. "Learning dynamic inter-farm dependencies for wind power forecasting via adaptive sparse graph attention network," Renewable Energy, Elsevier, vol. 258(C).
    4. Hanif, M.F. & Mi, J., 2024. "Harnessing AI for solar energy: Emergence of transformer models," Applied Energy, Elsevier, vol. 369(C).
    5. Hu, Ziqi & Li, Mingchen & Tang, Hao & Wang, Zhe, 2025. "AutoControl: An end-to-end fully automated workflow for control design of building energy systems," Energy, Elsevier, vol. 336(C).
    6. Cheng, Fang & Liu, Hui, 2024. "Multi-step electric vehicles charging loads forecasting: An autoformer variant with feature extraction, frequency enhancement, and error correction blocks," Applied Energy, Elsevier, vol. 376(PB).
    7. Zhang, Zongbin & Huang, Xiaoqiao & Li, Chengli & Cheng, Feiyan & Tai, Yonghang, 2025. "CRAformer: A cross-residual attention transformer for solar irradiation multistep forecasting," Energy, Elsevier, vol. 320(C).
    8. Nie, Yuhao & Paletta, Quentin & Scott, Andea & Pomares, Luis Martin & Arbod, Guillaume & Sgouridis, Sgouris & Lasenby, Joan & Brandt, Adam, 2024. "Sky image-based solar forecasting using deep learning with heterogeneous multi-location data: Dataset fusion versus transfer learning," Applied Energy, Elsevier, vol. 369(C).
    9. Tao, Kejun & Zhao, Jinghao & Tao, Ye & Qi, Qingqing & Tian, Yajun, 2024. "Operational day-ahead photovoltaic power forecasting based on transformer variant," Applied Energy, Elsevier, vol. 373(C).
    10. Wang, Chao & Lin, Hong & Yang, Ming & Fu, Xiaoling & Yuan, Yue & Wang, Zewei, 2024. "A novel chaotic time series wind power point and interval prediction method based on data denoising strategy and improved coati optimization algorithm," Chaos, Solitons & Fractals, Elsevier, vol. 187(C).
    11. Sun, Qihui & Yan, Feng & Sun, Wanqing & Zhou, Yuqing, 2025. "DWT-Former: Fusing wavelet-based multi-scale features and transformer-based temporal representations for photovoltaic power forecasting," Energy, Elsevier, vol. 341(C).
    12. Pei, Jingyin & Dong, Yunxuan & Guo, Pinghui & Wu, Thomas & Hu, Jianming, 2024. "A Hybrid Dual Stream ProbSparse Self-Attention Network for spatial–temporal photovoltaic power forecasting," Energy, Elsevier, vol. 305(C).
    13. Abad-Alcaraz, V. & Castilla, M. & Carballo, J.A. & Bonilla, J. & Álvarez, J.D., 2025. "Multimodal deep learning for solar radiation forecasting," Applied Energy, Elsevier, vol. 393(C).
    14. Khan, Zulfiqar Ahmad & Khan, Shabbir Ahmad & Hussain, Tanveer & Baik, Sung Wook, 2024. "DSPM: Dual sequence prediction model for efficient energy management in micro-grid," Applied Energy, Elsevier, vol. 356(C).
    15. Jiang, Wenjun & Liu, Bo & Liang, Yang & Gao, Huanxiang & Lin, Pengfei & Zhang, Dongqin & Hu, Gang, 2024. "Applicability analysis of transformer to wind speed forecasting by a novel deep learning framework with multiple atmospheric variables," Applied Energy, Elsevier, vol. 353(PB).
    16. Xie, Yang & Zheng, Jianyong & Mei, Fei & Taylor, Gareth & Gao, Ang, 2025. "An efficient approach for regional photovoltaic power forecasting optimization based on texture features from satellite images and transfer learning," Applied Energy, Elsevier, vol. 385(C).
    17. Zeng, Huanze & Wu, Binrong & Fang, Haoyu & Lin, Jiacheng, 2025. "Interpretable wind speed forecasting through two-stage decomposition with comprehensive relative importance analysis," Applied Energy, Elsevier, vol. 392(C).
    18. Ukwuoma, Chiagoziem C. & Cai, Dongsheng & Bamisile, Olusola & Yin, Hongbo & Nneji, Grace Ugochi & Monday, Happy N. & Oluwasanmi, Ariyo & Huang, Qi, 2024. "An attention fused sequence -to-sequence convolutional neural network for accurate solar irradiance forecasting and prediction using sky images," Renewable Energy, Elsevier, vol. 237(PB).
    19. Fan, Yuwei & Song, Tao & Feng, Chenlong & Liu, Chao & Jiang, Dongxiang, 2025. "Wind power prediction using foundation large time series models enhanced by time series prompt in exogenous and tuning forms," Applied Energy, Elsevier, vol. 400(C).
    20. Li, Na & Zhen, Zhao & Wang, Fei, 2026. "Surface solar irradiance retrieval approach based on satellite band optimal combination and multi-spatial scale mapping," Renewable Energy, Elsevier, vol. 256(PD).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:401:y:2025:i:pa:s030626192501400x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.