IDEAS home Printed from https://ideas.repec.org/a/wly/japmet/v34y2019i7p1050-1072.html
   My bibliography  Save this article

Macroeconomic forecast accuracy in a data‐rich environment

Author

Listed:
  • Rachidi Kotchoni
  • Maxime Leroux
  • Dalibor Stevanovic

Abstract

The performance of six classes of models in forecasting different types of economic series is evaluated in an extensive pseudo out‐of‐sample exercise. One of these forecasting models, regularized data‐rich model averaging (RDRMA), is new in the literature. The findings can be summarized in four points. First, RDRMA is difficult to beat in general and generates the best forecasts for real variables. This performance is attributed to the combination of regularization and model averaging, and it confirms that a smart handling of large data sets can lead to substantial improvements over univariate approaches. Second, the ARMA(1,1) model emerges as the best to forecast inflation changes in the short run, while RDRMA dominates at longer horizons. Third, the returns on the S&P 500 index are predictable by RDRMA at short horizons. Finally, the forecast accuracy and the optimal structure of the forecasting equations are quite unstable over time.

Suggested Citation

  • Rachidi Kotchoni & Maxime Leroux & Dalibor Stevanovic, 2019. "Macroeconomic forecast accuracy in a data‐rich environment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(7), pages 1050-1072, November.
  • Handle: RePEc:wly:japmet:v:34:y:2019:i:7:p:1050-1072
    DOI: 10.1002/jae.2725
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/jae.2725
    Download Restriction: no

    File URL: https://libkey.io/10.1002/jae.2725?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Banerjee, Anindya & Marcellino, Massimiliano & Masten, Igor, 2014. "Forecasting with factor-augmented error correction models," International Journal of Forecasting, Elsevier, vol. 30(3), pages 589-612.
    2. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    3. Breitung, Jörg & Eickmeier, Sandra, 2011. "Testing for structural breaks in dynamic factor models," Journal of Econometrics, Elsevier, vol. 163(1), pages 71-84, July.
    4. Pesaran, M Hashem & Timmermann, Allan, 1992. "A Simple Nonparametric Test of Predictive Performance," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 561-565, October.
    5. Boot, Tom & Nibbering, Didier, 2019. "Forecasting using random subspace methods," Journal of Econometrics, Elsevier, vol. 209(2), pages 391-406.
    6. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2015. "Bayesian VARs: Specification Choices and Forecast Accuracy," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(1), pages 46-73, January.
    7. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    8. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    9. Davide Pettenuzzo & Allan Timmermann, 2017. "Forecasting Macroeconomic Variables Under Model Instability," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(2), pages 183-201, April.
    10. Antonello D'Agostino & Luca Gambetti & Domenico Giannone, 2013. "Macroeconomic forecasting and structural change," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(1), pages 82-101, January.
    11. Gary M. Koop, 2013. "Forecasting with Medium and Large Bayesian VARS," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 177-203, March.
    12. Ben S. Bernanke & Jean Boivin & Piotr Eliasz, 2005. "Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 120(1), pages 387-422.
    13. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2013. "Complete subset regressions," Journal of Econometrics, Elsevier, vol. 177(2), pages 357-373.
    14. Raffaella Giacomini & Barbara Rossi, 2009. "Detecting and Predicting Forecast Breakdowns," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 669-705.
    15. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    16. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    17. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2015. "Prior Selection for Vector Autoregressions," The Review of Economics and Statistics, MIT Press, vol. 97(2), pages 436-451, May.
    18. Pierre Perron & Serena Ng, 1996. "Useful Modifications to some Unit Root Tests with Dependent Errors and their Local Asymptotic Properties," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 63(3), pages 435-463.
    19. Groen, Jan J.J. & Kapetanios, George, 2016. "Revisiting useful approaches to data-rich macroeconomic forecasting," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 221-239.
    20. Mao Takongmo, Charles Olivier & Stevanovic, Dalibor, 2015. "Selection Of The Number Of Factors In Presence Of Structural Instability: A Monte Carlo Study," L'Actualité Economique, Société Canadienne de Science Economique, vol. 91(1-2), pages 177-233, Mars-Juin.
    21. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, February.
    22. Kyle Jurado & Sydney C. Ludvigson & Serena Ng, 2015. "Measuring Uncertainty," American Economic Review, American Economic Association, vol. 105(3), pages 1177-1216, March.
    23. De Mol, Christine & Giannone, Domenico & Reichlin, Lucrezia, 2008. "Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components?," Journal of Econometrics, Elsevier, vol. 146(2), pages 318-328, October.
    24. Forni, Mario & Hallin, Marc & Lippi, Marco & Reichlin, Lucrezia, 2005. "The Generalized Dynamic Factor Model: One-Sided Estimation and Forecasting," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 830-840, September.
    25. Pierre Guérin & Danilo Leiva-Leon & Massimiliano Marcellino, 2020. "Markov-Switching Three-Pass Regression Filter," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 285-302, April.
    26. Claudia Foroni & Massimiliano Marcellino & Dalibor Stevanovic, 2019. "Mixed‐frequency models with moving‐average components," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(5), pages 688-706, August.
    27. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    28. Sandra Eickmeier & Wolfgang Lemke & Massimiliano Marcellino, 2015. "Classical time varying factor-augmented vector auto-regressive models—estimation, forecasting and structural analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(3), pages 493-533, June.
    29. Marta Bańbura, 2008. "Large Bayesian VARs," 2008 Meeting Papers 334, Society for Economic Dynamics.
    30. Del Negro, Marco & Hasegawa, Raiden B. & Schorfheide, Frank, 2016. "Dynamic prediction pools: An investigation of financial frictions and forecasting performance," Journal of Econometrics, Elsevier, vol. 192(2), pages 391-405.
    31. Boivin, Jean & Ng, Serena, 2006. "Are more data always better for factor analysis?," Journal of Econometrics, Elsevier, vol. 132(1), pages 169-194, May.
    32. Tatevik Sekhposyan & Barbara Rossi, 2008. "Has modelsí forecasting performance for US output growth and inflation changed over time, and when?," Working Papers 09-02, Duke University, Department of Economics.
    33. Graham Elliott & Allan Timmermann, 2005. "Optimal Forecast Combination Under Regime Switching ," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(4), pages 1081-1102, November.
    34. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    35. Xu Cheng & Zhipeng Liao & Frank Schorfheide, 2016. "Shrinkage Estimation of High-Dimensional Factor Models with Structural Instabilities," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 83(4), pages 1511-1543.
    36. Diebold, Francis X. & Shin, Minchul, 2019. "Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1679-1691.
    37. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.
    38. Jean Boivin & Serena Ng, 2005. "Understanding and Comparing Factor-Based Forecasts," International Journal of Central Banking, International Journal of Central Banking, vol. 1(3), December.
    39. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    40. Serena Ng & Pierre Perron, 2001. "LAG Length Selection and the Construction of Unit Root Tests with Good Size and Power," Econometrica, Econometric Society, vol. 69(6), pages 1519-1554, November.
    41. Rossi, Barbara & Sekhposyan, Tatevik, 2010. "Have economic models' forecasting performance for US output growth and inflation changed over time, and when?," International Journal of Forecasting, Elsevier, vol. 26(4), pages 808-835, October.
    42. Jean Boivin & Marc P. Giannoni, 2006. "Has Monetary Policy Become More Effective?," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 445-462, August.
    43. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    44. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    45. Jean-Marie Dufour & Dalibor Stevanović, 2013. "Factor-Augmented VARMA Models With Macroeconomic Applications," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(4), pages 491-506, October.
    46. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    47. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2015. "Complete subset regressions with large-dimensional sets of predictors," Journal of Economic Dynamics and Control, Elsevier, vol. 54(C), pages 86-110.
    48. David F. Hendry & Michael P. Clements, 2004. "Pooling of forecasts," Econometrics Journal, Royal Economic Society, vol. 7(1), pages 1-31, June.
    49. G. Elliott & C. Granger & A. Timmermann (ed.), 2013. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 2, number 2.
    50. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    51. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    2. Maxime Leroux & Rachidi Kotchoni & Dalibor Stevanovic, 2017. "Forecasting economic activity in data-rich environment," Working Papers hal-04141668, HAL.
    3. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    4. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    5. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    6. Andrea Carriero & Francesco Corsello & Massimiliano Marcellino, 2022. "The global component of inflation volatility," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(4), pages 700-721, June.
    7. Helmut Lütkepohl, 2014. "Structural Vector Autoregressive Analysis in a Data Rich Environment: A Survey," Discussion Papers of DIW Berlin 1351, DIW Berlin, German Institute for Economic Research.
    8. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    9. Berg, Tim O. & Henzel, Steffen R., 2015. "Point and density forecasts for the euro area using Bayesian VARs," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1067-1095.
    10. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    11. Pierre Guérin & Danilo Leiva-Leon & Massimiliano Marcellino, 2020. "Markov-Switching Three-Pass Regression Filter," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 285-302, April.
    12. Byron Botha & Rulof Burger & Kevin Kotzé & Neil Rankin & Daan Steenkamp, 2023. "Big data forecasting of South African inflation," Empirical Economics, Springer, vol. 65(1), pages 149-188, July.
    13. Dimitris Korobilis, 2018. "Machine Learning Macroeconometrics: A Primer," Working Paper series 18-30, Rimini Centre for Economic Analysis.
    14. Goulet Coulombe, Philippe & Leroux, Maxime & Stevanovic, Dalibor & Surprenant, Stéphane, 2021. "Macroeconomic data transformations matter," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1338-1354.
    15. Hauzenberger, Niko & Huber, Florian & Klieber, Karin, 2023. "Real-time inflation forecasting using non-linear dimension reduction techniques," International Journal of Forecasting, Elsevier, vol. 39(2), pages 901-921.
    16. Gupta, Rangan & Kabundi, Alain & Miller, Stephen M., 2011. "Forecasting the US real house price index: Structural and non-structural models with and without fundamentals," Economic Modelling, Elsevier, vol. 28(4), pages 2013-2021, July.
    17. Charles Rahal, 2015. "Housing Market Forecasting with Factor Combinations," Discussion Papers 15-05, Department of Economics, University of Birmingham.
    18. Yousuf, Kashif & Ng, Serena, 2021. "Boosting high dimensional predictive regressions with time varying parameters," Journal of Econometrics, Elsevier, vol. 224(1), pages 60-87.
    19. Pierre Guérin & Danilo Leiva-Leon & Massimiliano Marcellino, 2020. "Markov-Switching Three-Pass Regression Filter," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 285-302, April.
    20. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:japmet:v:34:y:2019:i:7:p:1050-1072. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.interscience.wiley.com/jpages/0883-7252/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.