IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2507.07477.html
   My bibliography  Save this paper

Electricity Market Predictability: Virtues of Machine Learning and Links to the Macroeconomy

Author

Listed:
  • Jinbo Cai
  • Wenze Li
  • Wenjie Wang

Abstract

With stakeholder-level in-market data, we conduct a comparative analysis of machine learning (ML) for forecasting electricity prices in Singapore, spanning 15 individual models and 4 ensemble approaches. Our empirical findings justify the three virtues of ML models: (1) the virtue of capturing non-linearity, (2) the complexity (Kelly et al., 2024) and (3) the l2-norm and bagging techniques in a weak factor environment (Shen and Xiu, 2024). Simulation also supports the first virtue. Penalizing prediction correlation improves ensemble performance when individual models are highly correlated. The predictability can be translated into sizable economic gains under the mean-variance framework. We also reveal significant patterns of time-series heterogeneous predictability across macro regimes: predictability is clustered in expansion, volatile market and extreme geopolitical risk periods. Our feature importance results agree with the complex dynamics of Singapore's electricity market after de regulation, yet highlight its relatively supply-driven nature with the continued presence of strong regulatory influences.

Suggested Citation

  • Jinbo Cai & Wenze Li & Wenjie Wang, 2025. "Electricity Market Predictability: Virtues of Machine Learning and Links to the Macroeconomy," Papers 2507.07477, arXiv.org.
  • Handle: RePEc:arx:papers:2507.07477
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2507.07477
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mirakyan, Atom & Meyer-Renschhausen, Martin & Koch, Andreas, 2017. "Composite forecasting approach, application for next-day electricity price forecasting," Energy Economics, Elsevier, vol. 66(C), pages 228-237.
    2. Dario Caldara & Matteo Iacoviello, 2022. "Measuring Geopolitical Risk," American Economic Review, American Economic Association, vol. 112(4), pages 1194-1225, April.
    3. Kelly, Bryan T. & Pruitt, Seth & Su, Yinan, 2019. "Characteristics are covariances: A unified model of risk and return," Journal of Financial Economics, Elsevier, vol. 134(3), pages 501-524.
    4. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    5. Dudek, Grzegorz, 2016. "Multilayer perceptron for GEFCom2014 probabilistic electricity price forecasting," International Journal of Forecasting, Elsevier, vol. 32(3), pages 1057-1060.
    6. Palm, F. & Zellner, A., 1991. "To combine or not to combine? issues of combining forecasts," LIDAM Discussion Papers CORE 1991022, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    7. Andrew W. Lo, A. Craig MacKinlay, 1988. "Stock Market Prices do not Follow Random Walks: Evidence from a Simple Specification Test," The Review of Financial Studies, Society for Financial Studies, vol. 1(1), pages 41-66.
    8. Stefan Nagel, 2012. "Evaporating Liquidity," The Review of Financial Studies, Society for Financial Studies, vol. 25(7), pages 2005-2039.
    9. Daniele Bianchi & Matthias Büchner & Andrea Tamoni, 2021. "Bond Risk Premiums with Machine Learning [Quadratic term structure models: Theory and evidence]," The Review of Financial Studies, Society for Financial Studies, vol. 34(2), pages 1046-1089.
    10. van der Stelt, Sander & AlSkaif, Tarek & van Sark, Wilfried, 2018. "Techno-economic analysis of household and community energy storage for residential prosumers with smart appliances," Applied Energy, Elsevier, vol. 209(C), pages 266-276.
    11. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    12. Knittel, Christopher R. & Roberts, Michael R., 2005. "An empirical examination of restructured electricity prices," Energy Economics, Elsevier, vol. 27(5), pages 791-817, September.
    13. Pettenuzzo, Davide & Timmermann, Allan & Valkanov, Rossen, 2014. "Forecasting stock returns under economic constraints," Journal of Financial Economics, Elsevier, vol. 114(3), pages 517-553.
    14. Florian Ziel & Rick Steinert, 2015. "Electricity Price Forecasting using Sale and Purchase Curves: The X-Model," Papers 1509.00372, arXiv.org, revised Aug 2016.
    15. Kwiatkowski, Denis & Phillips, Peter C. B. & Schmidt, Peter & Shin, Yongcheol, 1992. "Testing the null hypothesis of stationarity against the alternative of a unit root : How sure are we that economic time series have a unit root?," Journal of Econometrics, Elsevier, vol. 54(1-3), pages 159-178.
    16. Dashan Huang & Fuwei Jiang & Jun Tu & Guofu Zhou, 2015. "Investor Sentiment Aligned: A Powerful Predictor of Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 28(3), pages 791-837.
    17. Amit Goyal & Ivo Welch & Athanasse Zafirov, 2024. "A Comprehensive 2022 Look at the Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 37(11), pages 3490-3557.
    18. Christopher Kath & Florian Ziel, 2018. "The value of forecasts: Quantifying the economic gains of accurate quarter-hourly electricity price forecasts," Papers 1811.08604, arXiv.org.
    19. Ravi Jagannathan & Tongshu Ma, 2003. "Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps," Journal of Finance, American Finance Association, vol. 58(4), pages 1651-1683, August.
    20. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    21. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    22. John Y. Campbell, Robert J. Shiller, 1988. "The Dividend-Price Ratio and Expectations of Future Dividends and Discount Factors," The Review of Financial Studies, Society for Financial Studies, vol. 1(3), pages 195-228.
    23. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    24. Francis X. Diebold, 2015. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(1), pages 1-1, January.
    25. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    26. Fama, Eugene F. & Schwert, G. William, 1977. "Asset returns and inflation," Journal of Financial Economics, Elsevier, vol. 5(2), pages 115-146, November.
    27. Stambaugh, Robert F. & Yu, Jianfeng & Yuan, Yu, 2012. "The short of it: Investor sentiment and anomalies," Journal of Financial Economics, Elsevier, vol. 104(2), pages 288-302.
    28. Campbell, John Y. & Lo, Andrew W. & MacKinlay, A. Craig & Whitelaw, Robert F., 1998. "The Econometrics Of Financial Markets," Macroeconomic Dynamics, Cambridge University Press, vol. 2(4), pages 559-562, December.
    29. Ghoddusi, Hamed & Creamer, Germán G. & Rafizadeh, Nima, 2019. "Machine learning in energy economics and finance: A review," Energy Economics, Elsevier, vol. 81(C), pages 709-727.
    30. Dangl, Thomas & Halling, Michael, 2012. "Predictive regressions with time-varying coefficients," Journal of Financial Economics, Elsevier, vol. 106(1), pages 157-181.
    31. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    32. Stock, James H. & Watson, Mark W., 2006. "Forecasting with Many Predictors," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 10, pages 515-554, Elsevier.
    33. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    34. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    35. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    36. Nowotarski, Jakub & Raviv, Eran & Trück, Stefan & Weron, Rafał, 2014. "An empirical comparison of alternative schemes for combining electricity spot price forecasts," Energy Economics, Elsevier, vol. 46(C), pages 395-412.
    37. István Ábel & Pierre L. Siklos & István P. Székely, 1998. "Money and Finance in the Transition to a Market Economy," Books, Edward Elgar Publishing, number 830.
    38. Jeremiah Green & John R. M. Hand & X. Frank Zhang, 2017. "The Characteristics that Provide Independent Information about Average U.S. Monthly Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 30(12), pages 4389-4436.
    39. A. N. Pettitt, 1979. "A Non‐Parametric Approach to the Change‐Point Problem," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 28(2), pages 126-135, June.
    40. Ziel, Florian & Steinert, Rick, 2016. "Electricity price forecasting using sale and purchase curves: The X-Model," Energy Economics, Elsevier, vol. 59(C), pages 435-454.
    41. Leland E. Farmer & Lawrence Schmidt & Allan Timmermann, 2023. "Pockets of Predictability," Journal of Finance, American Finance Association, vol. 78(3), pages 1279-1341, June.
    42. Keles, Dogan & Genoese, Massimo & Möst, Dominik & Fichtner, Wolf, 2012. "Comparison of extended mean-reversion and time series models for electricity spot price simulation considering negative prices," Energy Economics, Elsevier, vol. 34(4), pages 1012-1032.
    43. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    44. Jian Chen & Guohao Tang & Guofu Zhou & Wu Zhu, 2025. "ChatGPT and Deepseek: Can They Predict the Stock Market and Macroeconomy?," Papers 2502.10008, arXiv.org.
    45. Ghasemi, A. & Shayeghi, H. & Moradzadeh, M. & Nooshyar, M., 2016. "A novel hybrid algorithm for electricity price and load forecasting in smart grids with demand-side management," Applied Energy, Elsevier, vol. 177(C), pages 40-59.
    46. Doron Avramov & Si Cheng & Lior Metzker, 2023. "Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability," Management Science, INFORMS, vol. 69(5), pages 2587-2619, May.
    47. Bordignon, Silvano & Bunn, Derek W. & Lisi, Francesco & Nan, Fany, 2013. "Combining day-ahead forecasts for British electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 88-103.
    48. Benedict Foo & Deng Yao Koh & Juan Pang Tan & Wenjie Wang, 2023. "FORECASTING SINGAPORE’s ECONOMY USING STATISTICAL LEARNING AND FACTOR MODELS," The Singapore Economic Review (SER), World Scientific Publishing Co. Pte. Ltd., vol. 68(02), pages 319-353.
    49. Sai, Wei & Pan, Zehua & Liu, Siyu & Jiao, Zhenjun & Zhong, Zheng & Miao, Bin & Chan, Siew Hwa, 2023. "Event-driven forecasting of wholesale electricity price and frequency regulation price using machine learning algorithms," Applied Energy, Elsevier, vol. 352(C).
    50. Johnsen, Tor Arnt, 2001. "Demand, generation and price in the Norwegian market for electric power," Energy Economics, Elsevier, vol. 23(3), pages 227-251, May.
    51. Kath, Christopher & Ziel, Florian, 2018. "The value of forecasts: Quantifying the economic gains of accurate quarter-hourly electricity price forecasts," Energy Economics, Elsevier, vol. 76(C), pages 411-423.
    52. Henkel, Sam James & Martin, J. Spencer & Nardari, Federico, 2011. "Time-varying short-horizon predictability," Journal of Financial Economics, Elsevier, vol. 99(3), pages 560-580, March.
    53. Rafal Weron, 2006. "Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach," HSC Books, Hugo Steinhaus Center, Wroclaw University of Science and Technology, number hsbook0601, December.
    54. Olabi, A.G. & Onumaegbu, C. & Wilberforce, Tabbi & Ramadan, Mohamad & Abdelkareem, Mohammad Ali & Al – Alami, Abdul Hai, 2021. "Critical review of energy storage systems," Energy, Elsevier, vol. 214(C).
    55. repec:bla:jfinan:v:58:y:2003:i:4:p:1651-1684 is not listed on IDEAS
    56. Bryan Kelly & Semyon Malamud & Kangying Zhou, 2024. "The Virtue of Complexity in Return Prediction," Journal of Finance, American Finance Association, vol. 79(1), pages 459-503, February.
    57. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    58. Abdul Ghani Olabi & Tabbi Wilberforce & Mohammad Ali Abdelkareem & Mohamad Ramadan, 2021. "Critical Review of Flywheel Energy Storage System," Energies, MDPI, vol. 14(8), pages 1-33, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Smith, Simon C., 2021. "International stock return predictability," International Review of Financial Analysis, Elsevier, vol. 78(C).
    3. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    4. Davide Pettenuzzo & Francesco Ravazzolo, 2016. "Optimal Portfolio Choice Under Decision‐Based Model Combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1312-1332, November.
    5. , & Stein, Tobias, 2021. "Equity premium predictability over the business cycle," CEPR Discussion Papers 16357, C.E.P.R. Discussion Papers.
    6. Pan, Zhiyuan & Pettenuzzo, Davide & Wang, Yudong, 2020. "Forecasting stock returns: A predictor-constrained approach," Journal of Empirical Finance, Elsevier, vol. 55(C), pages 200-217.
    7. Mykola Babiak & Jozef Barunik, 2020. "Deep Learning, Predictability, and Optimal Portfolio Returns," CERGE-EI Working Papers wp677, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    8. Zhang, Hongwei & Wang, Wentao & Niu, Zibo, 2024. "Geopolitical risks and crude oil futures volatility: Evidence from machine learning," Resources Policy, Elsevier, vol. 98(C).
    9. Wang, Yudong & Pan, Zhiyuan & Liu, Li & Wu, Chongfeng, 2019. "Oil price increases and the predictability of equity premium," Journal of Banking & Finance, Elsevier, vol. 102(C), pages 43-58.
    10. Zhao, Albert Bo & Cheng, Tingting, 2022. "Stock return prediction: Stacking a variety of models," Journal of Empirical Finance, Elsevier, vol. 67(C), pages 288-317.
    11. Xue Gong & Weiguo Zhang & Yuan Zhao & Xin Ye, 2023. "Forecasting stock volatility with a large set of predictors: A new forecast combination method," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1622-1647, November.
    12. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    13. Dichtl, Hubert & Drobetz, Wolfgang & Neuhierl, Andreas & Wendt, Viktoria-Sophie, 2021. "Data snooping in equity premium prediction," International Journal of Forecasting, Elsevier, vol. 37(1), pages 72-94.
    14. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    15. Faria, Gonçalo & Verona, Fabio, 2024. "Enhancing forecast accuracy through frequencydomain combination: Applications to financial and economic indicators," Bank of Finland Research Discussion Papers 14/2024, Bank of Finland.
    16. Nonejad, Nima, 2023. "Conditional out-of-sample predictability of aggregate equity returns and aggregate equity return volatility using economic variables," Journal of Empirical Finance, Elsevier, vol. 70(C), pages 91-122.
    17. Grzegorz Marcjasz & Tomasz Serafin & Rafał Weron, 2018. "Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 11(9), pages 1-20, September.
    18. Nonejad, Nima, 2022. "Predicting equity premium out-of-sample by conditioning on newspaper-based uncertainty measures: A comparative study," International Review of Financial Analysis, Elsevier, vol. 83(C).
    19. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    20. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2507.07477. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.