IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v356y2024ics0306261923017981.html
   My bibliography  Save this article

Can end-to-end data-driven models outperform traditional semi-physical models in separating 1-min irradiance?

Author

Listed:
  • Chu, Yinghao
  • Yang, Dazhi
  • Yu, Hanxin
  • Zhao, Xin
  • Li, Mengying

Abstract

As a crucial component of the model chain, which facilitates irradiance-to-power conversion during solar resource assessment and forecasting, separation modeling continues to draw attention in both academia and industry. However, when evaluating even the best separation model today, one can quickly recognize its limited accuracy compared to other energy meteorology models such as transposition models. The task of separating global horizontal irradiance into diffuse and beam components does not seem soluble by any derivative effort aimed at tweaking the existing semi-physical models. As a result, an appealing alternative is to consider end-to-end data-driven models, which have demonstrated predictive capability in scenarios where the volume of data is substantial and the interaction among variables is complex. This work discusses the separation of 1-min irradiance from a data-driven perspective. In this preliminary study, a total of 10 representative data-driven separation models are developed and compared to the state-of-the-art semi-physical models, using a comprehensive 1-min irradiance database that spans five years and covers numerous climate types. The average error of the data-driven models is found to be 15.2% to 22.6% lower than that of the semi-physical models for training locations and 7.9% to 17.6% lower for completely unseen locations. Data-driven models also have significantly lower standard deviations (up to 87.2% even for completely unseen locations), highlighting their robustness. In addition, this work provides a guideline for choosing between data-driven and semi-physical models based on data availability, application needs, computational resources, interpretability, and model adaptability. Furthermore, the study underscores the challenges in accurately predicting the diffuse fraction using available input features and indicates that the incorporation of additional weather-related variables and domain knowledge could enhance the performance of data-driven separation models.

Suggested Citation

  • Chu, Yinghao & Yang, Dazhi & Yu, Hanxin & Zhao, Xin & Li, Mengying, 2024. "Can end-to-end data-driven models outperform traditional semi-physical models in separating 1-min irradiance?," Applied Energy, Elsevier, vol. 356(C).
    • Handle: RePEc:eee:appene:v:356:y:2024:i:c:s0306261923017981
    DOI: 10.1016/j.apenergy.2023.122434
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261923017981
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.122434?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pedro, Hugo T.C. & Coimbra, Carlos F.M. & David, Mathieu & Lauret, Philippe, 2018. "Assessment of machine learning techniques for deterministic and probabilistic intra-hour solar forecasts," Renewable Energy, Elsevier, vol. 123(C), pages 191-203.
    2. Mayer, Martin János, 2022. "Benefits of physical and machine learning hybridization for photovoltaic power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    3. Every, Jeremy P. & Li, Li & Dorrell, David G., 2020. "Köppen-Geiger climate classification adjustment of the BRL diffuse irradiation model for Australian locations," Renewable Energy, Elsevier, vol. 147(P1), pages 2453-2469.
    4. Sengupta, Manajit & Xie, Yu & Lopez, Anthony & Habte, Aron & Maclaurin, Galen & Shelby, James, 2018. "The National Solar Radiation Data Base (NSRDB)," Renewable and Sustainable Energy Reviews, Elsevier, vol. 89(C), pages 51-60.
    5. Chu, Yinghao & Coimbra, Carlos F.M., 2017. "Short-term probabilistic forecasts for Direct Normal Irradiance," Renewable Energy, Elsevier, vol. 101(C), pages 526-536.
    6. Yagli, Gokhan Mert & Yang, Dazhi & Gandhi, Oktoviano & Srinivasan, Dipti, 2020. "Can we justify producing univariate machine-learning forecasts with satellite-derived solar irradiance?," Applied Energy, Elsevier, vol. 259(C).
    7. Mayer, Martin János & Yang, Dazhi, 2022. "Probabilistic photovoltaic power forecasting using a calibrated ensemble of model chains," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    8. Yang, Dazhi, 2022. "Estimating 1-min beam and diffuse irradiance from the global irradiance: A review and an extensive worldwide comparison of latest separation models at 126 stations," Renewable and Sustainable Energy Reviews, Elsevier, vol. 159(C).
    9. Yang, Dazhi & Gu, Yizhan & Mayer, Martin János & Gueymard, Christian A. & Wang, Wenting & Kleissl, Jan & Li, Mengying & Chu, Yinghao & Bright, Jamie M., 2024. "Regime-dependent 1-min irradiance separation model with climatology clustering," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    10. Starke, Allan R. & Lemos, Leonardo F.L. & Boland, John & Cardemil, José M. & Colle, Sergio, 2018. "Resolution of the cloud enhancement problem for one-minute diffuse radiation prediction," Renewable Energy, Elsevier, vol. 125(C), pages 472-484.
    11. Anagnostos, D. & Schmidt, T. & Cavadias, S. & Soudris, D. & Poortmans, J. & Catthoor, F., 2019. "A method for detailed, short-term energy yield forecasting of photovoltaic installations," Renewable Energy, Elsevier, vol. 130(C), pages 122-129.
    12. Mayer, Martin János & Yang, Dazhi, 2023. "Calibration of deterministic NWP forecasts and its impact on verification," International Journal of Forecasting, Elsevier, vol. 39(2), pages 981-991.
    13. Ridley, Barbara & Boland, John & Lauret, Philippe, 2010. "Modelling of diffuse solar fraction with multiple predictors," Renewable Energy, Elsevier, vol. 35(2), pages 478-483.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Dazhi & Gu, Yizhan & Mayer, Martin János & Gueymard, Christian A. & Wang, Wenting & Kleissl, Jan & Li, Mengying & Chu, Yinghao & Bright, Jamie M., 2024. "Regime-dependent 1-min irradiance separation model with climatology clustering," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    2. Yang, Dazhi, 2022. "Estimating 1-min beam and diffuse irradiance from the global irradiance: A review and an extensive worldwide comparison of latest separation models at 126 stations," Renewable and Sustainable Energy Reviews, Elsevier, vol. 159(C).
    3. Mayer, Martin János & Yang, Dazhi, 2023. "Pairing ensemble numerical weather prediction with ensemble physical model chain for probabilistic photovoltaic power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 175(C).
    4. Manni, Mattia & Jouttijärvi, Sami & Ranta, Samuli & Miettunen, Kati & Lobaccaro, Gabriele, 2024. "Validation of model chains for global tilted irradiance on East-West vertical bifacial photovoltaics at high latitudes," Renewable Energy, Elsevier, vol. 220(C).
    5. Mayer, Martin János & Yang, Dazhi, 2022. "Probabilistic photovoltaic power forecasting using a calibrated ensemble of model chains," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    6. Mayer, Martin János & Yang, Dazhi & Szintai, Balázs, 2023. "Comparing global and regional downscaled NWP models for irradiance and photovoltaic power forecasting: ECMWF versus AROME," Applied Energy, Elsevier, vol. 352(C).
    7. Mayer, Martin János, 2022. "Benefits of physical and machine learning hybridization for photovoltaic power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    8. Nie, Yuhao & Li, Xiatong & Paletta, Quentin & Aragon, Max & Scott, Andea & Brandt, Adam, 2024. "Open-source sky image datasets for solar forecasting with deep learning: A comprehensive survey," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    9. Yagli, Gokhan Mert & Yang, Dazhi & Srinivasan, Dipti, 2019. "Automatic hourly solar forecasting using machine learning models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 105(C), pages 487-498.
    10. Ruiz-Arias, José A., 2023. "SPARTA: Solar parameterization for the radiative transfer of the cloudless atmosphere," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    11. Yang, Dazhi & Kleissl, Jan, 2023. "Summarizing ensemble NWP forecasts for grid operators: Consistency, elicitability, and economic value," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1640-1654.
    12. Simone Sala & Alfonso Amendola & Sonia Leva & Marco Mussetta & Alessandro Niccolai & Emanuele Ogliari, 2019. "Comparison of Data-Driven Techniques for Nowcasting Applied to an Industrial-Scale Photovoltaic Plant," Energies, MDPI, vol. 12(23), pages 1-19, November.
    13. Wang, Wenting & Yang, Dazhi & Huang, Nantian & Lyu, Chao & Zhang, Gang & Han, Xueying, 2022. "Irradiance-to-power conversion based on physical model chain: An application on the optimal configuration of multi-energy microgrid in cold climate," Renewable and Sustainable Energy Reviews, Elsevier, vol. 161(C).
    14. Zhang, Gang & Yang, Dazhi & Galanis, George & Androulakis, Emmanouil, 2022. "Solar forecasting with hourly updated numerical weather prediction," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    15. Yagli, Gokhan Mert & Yang, Dazhi & Srinivasan, Dipti, 2022. "Ensemble solar forecasting and post-processing using dropout neural network and information from neighboring satellite pixels," Renewable and Sustainable Energy Reviews, Elsevier, vol. 155(C).
    16. Starke, Allan R. & Lemos, Leonardo F.L. & Barni, Cristian M. & Machado, Rubinei D. & Cardemil, José M. & Boland, John & Colle, Sergio, 2021. "Assessing one-minute diffuse fraction models based on worldwide climate features," Renewable Energy, Elsevier, vol. 177(C), pages 700-714.
    17. Shi, Hongrong & Yang, Dazhi & Wang, Wenting & Fu, Disong & Gao, Ling & Zhang, Jinqiang & Hu, Bo & Shan, Yunpeng & Zhang, Yingjie & Bian, Yuxuan & Chen, Hongbin & Xia, Xiangao, 2023. "First estimation of high-resolution solar photovoltaic resource maps over China with Fengyun-4A satellite and machine learning," Renewable and Sustainable Energy Reviews, Elsevier, vol. 184(C).
    18. Jiang, Chengcheng & Zhu, Qunzhi, 2023. "Evaluating the most significant input parameters for forecasting global solar radiation of different sequences based on Informer," Applied Energy, Elsevier, vol. 348(C).
    19. Rodríguez-Benítez, Francisco J. & López-Cuesta, Miguel & Arbizu-Barrena, Clara & Fernández-León, María M. & Pamos-Ureña, Miguel Á. & Tovar-Pescador, Joaquín & Santos-Alamillos, Francisco J. & Pozo-Váz, 2021. "Assessment of new solar radiation nowcasting methods based on sky-camera and satellite imagery," Applied Energy, Elsevier, vol. 292(C).
    20. Zhang, Liwenbo & Wilson, Robin & Sumner, Mark & Wu, Yupeng, 2023. "Advanced multimodal fusion method for very short-term solar irradiance forecasting using sky images and meteorological data: A gate and transformer mechanism approach," Renewable Energy, Elsevier, vol. 216(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:356:y:2024:i:c:s0306261923017981. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.