IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2602.16527.html

Model selection confidence sets for time series models with applications to electricity load data

Author

Listed:
  • Piersilvio De Bortoli
  • Davide Ferrari
  • Francesco Ravazzolo
  • Luca Rossini

Abstract

This paper studies the Model Selection Confidence Set (MSCS) methodology for univariate time series models involving autoregressive and moving average components, and applies it to study model selection uncertainty in the Italian electricity load data. Rather than relying on a single model selected by an arbitrary criterion, the MSCS identifies a set of models that are statistically indistinguishable from the true data-generating process at a given confidence level. The size and composition of this set reveal crucial information about model selection uncertainty: noisy data scenarios produce larger sets with many candidate models, while more informative cases narrow the set considerably. To study the importance of each model term, we consider numerical statistics measuring the frequency with which each term is included in both the entire MSCS and in Lower Boundary Models (LBM), its most parsimonious specifications. Applied to Italian hourly electricity load data, the MSCS methodology reveals marked intraday variation in model selection uncertainty and isolates a collection of model specifications that deliver competitive short-term forecasts while highlighting key drivers of electricity load like intraday hourly lags, temperature, calendar effects and solar energy generation.

Suggested Citation

  • Piersilvio De Bortoli & Davide Ferrari & Francesco Ravazzolo & Luca Rossini, 2026. "Model selection confidence sets for time series models with applications to electricity load data," Papers 2602.16527, arXiv.org.
  • Handle: RePEc:arx:papers:2602.16527
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2602.16527
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yang Li & Yuetian Luo & Davide Ferrari & Xiaonan Hu & Yichen Qin, 2019. "Rejoinder to Discussions on: Model confidence bounds for variable selection," Biometrics, The International Biometric Society, vol. 75(2), pages 411-413, June.
    2. Jie Ding & Vahid Tarokh & Yuhong Yang, 2018. "Model Selection Techniques -- An Overview," Papers 1810.09583, arXiv.org.
    3. Taylor, James W., 2010. "Triple seasonal methods for short-term electricity demand forecasting," European Journal of Operational Research, Elsevier, vol. 204(1), pages 139-152, July.
    4. Haben, Stephen & Giasemidis, Georgios & Ziel, Florian & Arora, Siddharth, 2019. "Short term load forecasting and the effect of temperature at the low voltage level," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1469-1484.
    5. Dordonnat, V. & Koopman, S.J. & Ooms, M. & Dessertaine, A. & Collet, J., 2008. "An hourly periodic state space model for modelling French national electricity load," International Journal of Forecasting, Elsevier, vol. 24(4), pages 566-587.
    6. Hong, Tao & Wang, Pu & White, Laura, 2015. "Weather station selection for electric load forecasting," International Journal of Forecasting, Elsevier, vol. 31(2), pages 286-295.
    7. Jiahua Chen & Zehua Chen, 2008. "Extended Bayesian information criteria for model selection with large model spaces," Biometrika, Biometrika Trust, vol. 95(3), pages 759-771.
    8. Henley, Andrew & Peirson, John, 1997. "Non-linearities in Electricity Demand and Temperature: Parametric versus Non-parametric Methods," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 59(1), pages 149-162, February.
    9. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    10. Kasun Bandara & Rob J. Hyndman & Christoph Bergmeir, 2025. "MSTL: a seasonal-trend decomposition algorithm for time series with multiple seasonal patterns," International Journal of Operational Research, Inderscience Enterprises Ltd, vol. 52(1), pages 79-98.
    11. Wang, Qin & Tuohy, Aidan & Ortega-Vazquez, Miguel & Bello, Mobolaji & Ela, Erik & Kirk-Davidoff, Daniel & Hobbs, William B. & Ault, David J. & Philbrick, Russ, 2023. "Quantifying the value of probabilistic forecasting for power system operation planning," Applied Energy, Elsevier, vol. 343(C).
    12. Kang, Jieyi & Reiner, David M., 2022. "What is the effect of weather on household electricity consumption? Empirical evidence from Ireland," Energy Economics, Elsevier, vol. 111(C).
    13. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    14. Cao, Chaojin & He, Yaoyao & Yang, Xiaodong, 2025. "Online decoupling feature framework for optimal probabilistic load forecasting in concept drift environments," Applied Energy, Elsevier, vol. 392(C).
    15. Yang Li & Yuetian Luo & Davide Ferrari & Xiaonan Hu & Yichen Qin, 2019. "Model confidence bounds for variable selection," Biometrics, The International Biometric Society, vol. 75(2), pages 392-403, June.
    16. Enrich, Jacint & Li, Ruoyi & Mizrahi, Alejandro & Reguant, Mar, 2024. "Measuring the impact of time-of-use pricing on electricity consumption: Evidence from Spain," Journal of Environmental Economics and Management, Elsevier, vol. 123(C).
    17. R. M. Kuiper & H. Hoijtink & M. J. Silvapulle, 2011. "An Akaike-type information criterion for model selection under inequality constraints," Biometrika, Biometrika Trust, vol. 98(2), pages 495-501.
    18. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    19. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    20. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    21. Wang, Pu & Liu, Bidong & Hong, Tao, 2016. "Electric load forecasting with recency effect: A big data approach," International Journal of Forecasting, Elsevier, vol. 32(3), pages 585-597.
    22. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    23. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    24. J W Taylor, 2003. "Short-term electricity demand forecasting using double seasonal exponential smoothing," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(8), pages 799-805, August.
    25. Arie Preminger & David Wettstein, 2005. "Using the Penalized Likelihood Method for Model Selection with Nuisance Parameters Present only under the Alternative: An Application to Switching Regression Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 26(5), pages 715-741, September.
    26. Bashiri Behmiri, Niaz & Fezzi, Carlo & Ravazzolo, Francesco, 2023. "Incorporating air temperature into mid-term electricity load forecasting models using time-series regressions and neural networks," Energy, Elsevier, vol. 278(C).
    27. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    28. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    29. Newey, Whitney K, 1985. "Maximum Likelihood Specification Testing and Conditional Moment Tests," Econometrica, Econometric Society, vol. 53(5), pages 1047-1070, September.
    30. Fezzi, Carlo & Fanghella, Valeria, 2021. "Tracking GDP in real-time using electricity market data: Insights from the first wave of COVID-19 across Europe," European Economic Review, Elsevier, vol. 139(C).
    31. Yoshimasa Uematsu & Shinya Tanaka, 2019. "High†dimensional macroeconomic forecasting and variable selection via penalized regression," The Econometrics Journal, Royal Economic Society, vol. 22(1), pages 34-56.
    32. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Qin, Yichen & Wang, Linna & Li, Yang & Li, Rong, 2023. "Visualization and assessment of model selection uncertainty," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    4. Mekelburg, Erik & Strauss, Jack, 2024. "Pooling and winsorizing machine learning forecasts to predict stock returns with high-dimensional data," Journal of Empirical Finance, Elsevier, vol. 79(C).
    5. Barbara Rossi, 2021. "Forecasting in the Presence of Instabilities: How We Know Whether Models Predict Well and How to Improve Them," Journal of Economic Literature, American Economic Association, vol. 59(4), pages 1135-1190, December.
    6. Xue Gong & Weiguo Zhang & Yuan Zhao & Xin Ye, 2023. "Forecasting stock volatility with a large set of predictors: A new forecast combination method," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1622-1647, November.
    7. Dumitrescu, Elena & Hué, Sullivan & Hurlin, Christophe & Tokpavi, Sessi, 2022. "Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects," European Journal of Operational Research, Elsevier, vol. 297(3), pages 1178-1192.
    8. Mogliani, Matteo & Simoni, Anna, 2021. "Bayesian MIDAS penalized regressions: Estimation, selection, and prediction," Journal of Econometrics, Elsevier, vol. 222(1), pages 833-860.
    9. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    10. Li, Jiahan & Chen, Weiye, 2014. "Forecasting macroeconomic time series: LASSO-based approaches and their forecast combinations with dynamic factor models," International Journal of Forecasting, Elsevier, vol. 30(4), pages 996-1015.
    11. Matthias Pelster & Johannes Vilsmeier, 2018. "The determinants of CDS spreads: evidence from the model space," Review of Derivatives Research, Springer, vol. 21(1), pages 63-118, April.
    12. Cepni, Oguzhan & Clements, Michael P., 2024. "How local is the local inflation factor? Evidence from emerging European countries," International Journal of Forecasting, Elsevier, vol. 40(1), pages 160-183.
    13. Li, Zhao-Chen & Xie, Chi & Wang, Gang-Jin & Zhu, You & Zeng, Zhi-Jian & Gong, Jue, 2024. "Forecasting global stock market volatilities: A shrinkage heterogeneous autoregressive (HAR) model with a large cross-market predictor set," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 673-711.
    14. Ardia, David & Bluteau, Keven & Boudt, Kris, 2019. "Questioning the news about economic growth: Sparse forecasting using thousands of news-based sentiment values," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1370-1386.
    15. Zhang, Hongwei & Wang, Wentao & Niu, Zibo, 2024. "Geopolitical risks and crude oil futures volatility: Evidence from machine learning," Resources Policy, Elsevier, vol. 98(C).
    16. Borup, Daniel & Rapach, David E. & Schütte, Erik Christian Montes, 2023. "Mixed-frequency machine learning: Nowcasting and backcasting weekly initial claims with daily internet search volume data," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1122-1144.
    17. George Milunovich, 2020. "Forecasting Australia's real house price index: A comparison of time series and machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(7), pages 1098-1118, November.
    18. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    19. Chuliá, Helena & Garrón, Ignacio & Uribe, Jorge M., 2024. "Daily growth at risk: Financial or real drivers? The answer is not always the same," International Journal of Forecasting, Elsevier, vol. 40(2), pages 762-776.
    20. Joseph, Andreas & Potjagailo, Galina & Chakraborty, Chiranjit & Kapetanios, George, 2024. "Forecasting UK inflation bottom up," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1521-1538.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2602.16527. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.