IDEAS home Printed from
   My bibliography  Save this article

An Empirical Comparison of Machine Learning Models for Time Series Forecasting


  • Nesreen Ahmed
  • Amir Atiya
  • Neamat El Gayar
  • Hisham El-Shishiny


In this work we present a large scale comparison study for the major machine learning models for time series forecasting. Specifically, we apply the models on the monthly M3 time series competition data (around a thousand time series). There have been very few, if any, large scale comparison studies for machine learning models for the regression or the time series forecasting problems, so we hope this study would fill this gap. The models considered are multilayer perceptron, Bayesian neural networks, radial basis functions, generalized regression neural networks (also called kernel regression), K-nearest neighbor regression, CART regression trees, support vector regression, and Gaussian processes. The study reveals significant differences between the different methods. The best two methods turned out to be the multilayer perceptron and the Gaussian process regression. In addition to model comparisons, we have tested different preprocessing methods and have shown that they have different impacts on the performance.

Suggested Citation

  • Nesreen Ahmed & Amir Atiya & Neamat El Gayar & Hisham El-Shishiny, 2010. "An Empirical Comparison of Machine Learning Models for Time Series Forecasting," Econometric Reviews, Taylor & Francis Journals, vol. 29(5-6), pages 594-621.
  • Handle: RePEc:taf:emetrv:v:29:y:2010:i:5-6:p:594-621 DOI: 10.1080/07474938.2010.481556

    Download full text from publisher

    File URL:
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Giacomini, Raffaella & Komunjer, Ivana, 2005. "Evaluation and Combination of Conditional Quantile Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 416-431, October.
    2. Palm, F. & Zellner, A., 1991. "To combine or not to combine? issues of combining forecasts," CORE Discussion Papers 1991022, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    3. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    4. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    5. repec:cor:louvrp:-1027 is not listed on IDEAS
    6. Diebold, Francis X., 1989. "Forecast combination and encompassing: Reconciling two divergent literatures," International Journal of Forecasting, Elsevier, vol. 5(4), pages 589-592.
    7. Shen, Xiaotong & Huang, Hsin-Cheng, 2006. "Optimal Model Assessment, Selection, and Combination," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 554-568, June.
    8. Stock, James H. & Watson, Mark W., 1999. "Forecasting inflation," Journal of Monetary Economics, Elsevier, vol. 44(2), pages 293-335, October.
    9. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," Review of Economic Studies, Oxford University Press, vol. 53(4), pages 671-690.
    10. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    11. Lewellen, Jonathan, 2004. "Predicting returns with financial ratios," Journal of Financial Economics, Elsevier, vol. 74(2), pages 209-235, November.
    12. Deutsch, Melinda & Granger, Clive W. J. & Terasvirta, Timo, 1994. "The combination of forecasts using changing weights," International Journal of Forecasting, Elsevier, vol. 10(1), pages 47-57, June.
    13. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    14. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    15. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    16. Li Fuchun & Tkacz Greg, 2004. "Combining Forecasts with Nonparametric Kernel Regressions," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 8(4), pages 1-18, December.
    17. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    18. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    19. Hibon, Michele & Evgeniou, Theodoros, 2005. "To combine or not to combine: selecting among forecasts and their combinations," International Journal of Forecasting, Elsevier, vol. 21(1), pages 15-24.
    20. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    21. Coulson, N.E. & Robins, R.P., 1989. "Forecast Combination In A Dynamic Setting," Papers 8-88-4, Pennsylvania State - Department of Economics.
    22. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    23. David F. Hendry & Michael P. Clements, 2004. "Pooling of forecasts," Econometrics Journal, Royal Economic Society, vol. 7(1), pages 1-31, June.
    24. Yeung Lewis Chan & James H. Stock & Mark W. Watson, 1999. "A dynamic factor model framework for forecast combination," Spanish Economic Review, Springer;Spanish Economic Association, vol. 1(2), pages 91-121.
    25. Diebold, Francis X. & Pauly, Peter, 1990. "The use of prior information in forecast combination," International Journal of Forecasting, Elsevier, vol. 6(4), pages 503-508, December.
    26. Engle, Robert F. & Granger, C. W. J. & Kraft, Dennis, 1984. "Combining competing forecasts of inflation using a bivariate arch model," Journal of Economic Dynamics and Control, Elsevier, vol. 8(2), pages 151-165, November.
    27. Jeremy Smith & Kenneth F. Wallis, 2009. "A Simple Explanation of the Forecast Combination Puzzle," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 331-355, June.
    28. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    29. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Green, Gareth & Richards, Timothy, 2016. "Interpreting Results of Demand Estimation from Machine Learning Models," 2016 Annual Meeting, July 31-August 2, 2016, Boston, Massachusetts 236147, Agricultural and Applied Economics Association.
    2. Cui, Can & Wu, Teresa & Hu, Mengqi & Weir, Jeffery D. & Li, Xiwang, 2016. "Short-term building energy model recommendation system: A meta-learning approach," Applied Energy, Elsevier, vol. 172(C), pages 251-263.
    3. Kock, Anders Bredahl & Teräsvirta, Timo, 2014. "Forecasting performances of three automated modelling techniques during the economic crisis 2007–2009," International Journal of Forecasting, Elsevier, vol. 30(3), pages 616-631.
    4. Andrawis, Robert R. & Atiya, Amir F. & El-Shishiny, Hisham, 2011. "Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition," International Journal of Forecasting, Elsevier, vol. 27(3), pages 672-688, July.
    5. Fischer, Thomas & Krauss, Christopher & Treichel, Alex, 2018. "Machine learning for time series forecasting - a simulation study," FAU Discussion Papers in Economics 02/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    6. Advait Sarkar & Neal Lathia & Cecilia Mascolo, 2015. "Comparing cities’ cycling patterns using online shared bicycle maps," Transportation, Springer, vol. 42(4), pages 541-559, July.
    7. repec:eee:touman:v:48:y:2015:i:c:p:64-72 is not listed on IDEAS
    8. Oscar Claveria & Enric Monte & Salvador Torra, 2016. "Modelling cross-dependencies between Spain’s regional tourism markets with an extension of the Gaussian process regression model," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 7(3), pages 341-357, August.
    9. Oscar Claveria & Enric Monte & Salvador Torra, 2017. "“Regional tourism demand forecasting with machine learning models: Gaussian process regression vs. neural network models in a multiple-input multiple-output setting"," IREA Working Papers 201701, University of Barcelona, Research Institute of Applied Economics, revised Jan 2017.
    10. Wu, Shaomin & Akbarov, Artur, 2011. "Support vector regression for warranty claim forecasting," European Journal of Operational Research, Elsevier, vol. 213(1), pages 196-204, August.
    11. Neil Kellard & Denise Osborn & Jerry Coakley & Imanol Arrieta-ibarra & Ignacio N. Lobato, 2015. "Testing for Predictability in Financial Returns Using Statistical Learning Procedures," Journal of Time Series Analysis, Wiley Blackwell, vol. 36(5), pages 672-686, September.
    12. repec:spr:waterr:v:32:y:2018:i:1:d:10.1007_s11269-017-1797-0 is not listed on IDEAS


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:emetrv:v:29:y:2010:i:5-6:p:594-621. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.