IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2203.06540.html
   My bibliography  Save this paper

Improving Macroeconomic Model Validity and Forecasting Performance with Pooled Country Data using Structural, Reduced Form, and Neural Network Model

Author

Listed:
  • Cameron Fen
  • Samir Undavia

Abstract

We show that pooling countries across a panel dimension to macroeconomic data can improve by a statistically significant margin the generalization ability of structural, reduced form, and machine learning (ML) methods to produce state-of-the-art results. Using GDP forecasts evaluated on an out-of-sample test set, this procedure reduces root mean squared error by 12\% across horizons and models for certain reduced-form models and by 24\% across horizons for dynamic structural general equilibrium models. Removing US data from the training set and forecasting out-of-sample country-wise, we show that reduced-form and structural models are more policy-invariant when trained on pooled data, and outperform a baseline that uses US data only. Given the comparative advantage of ML models in a data-rich regime, we demonstrate that our recurrent neural network model and automated ML approach outperform all tested baseline economic models. Robustness checks indicate that our outperformance is reproducible, numerically stable, and generalizable across models.

Suggested Citation

  • Cameron Fen & Samir Undavia, 2022. "Improving Macroeconomic Model Validity and Forecasting Performance with Pooled Country Data using Structural, Reduced Form, and Neural Network Model," Papers 2203.06540, arXiv.org.
  • Handle: RePEc:arx:papers:2203.06540
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2203.06540
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1059-1087.
    2. Bai, Jushan & Ng, Serena, 2010. "Instrumental Variable Estimation In A Data Rich Environment," Econometric Theory, Cambridge University Press, vol. 26(6), pages 1577-1606, December.
    3. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    4. Koop, Gary & Leon-Gonzalez, Roberto & Strachan, Rodney, 2012. "Bayesian model averaging in the instrumental variable regression model," Journal of Econometrics, Elsevier, vol. 171(2), pages 237-250.
    5. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    6. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    7. Anindya Banerjee & Massimiliano Marcellino & Chiara Osbat, 2004. "Some cautions on the use of panel methods for integrated series of macroeconomic data," Econometrics Journal, Royal Economic Society, vol. 7(2), pages 322-340, December.
    8. Pesaran, M Hashem & Timmermann, Allan, 1992. "A Simple Nonparametric Test of Predictive Performance," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 561-565, October.
    9. Pesaran, M. Hashem & Smith, Ron, 1995. "Estimating long-run relationships from dynamic heterogeneous panels," Journal of Econometrics, Elsevier, vol. 68(1), pages 79-113, July.
    10. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    11. Wright, Jonathan H., 2008. "Bayesian Model Averaging and exchange rate forecasts," Journal of Econometrics, Elsevier, vol. 146(2), pages 329-341, October.
    12. Badi H. Baltagi, 2008. "Forecasting with panel data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(2), pages 153-173.
    13. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    14. Doran, Howard E. & Schmidt, Peter, 2006. "GMM estimators with improved finite sample properties using principal components of the weighting matrix, with an application to the dynamic panel data model," Journal of Econometrics, Elsevier, vol. 133(1), pages 387-409, July.
    15. Thomas R. Cook & Aaron Smalter Hall, 2017. "Macroeconomic Indicator Forecasting with Deep Neural Networks," Research Working Paper RWP 17-11, Federal Reserve Bank of Kansas City.
    16. Wataru Miyamoto & Thuy Lan Nguyen & Dmitriy Sergeyev, 2018. "Government Spending Multipliers under the Zero Lower Bound: Evidence from Japan," American Economic Journal: Macroeconomics, American Economic Association, vol. 10(3), pages 247-277, July.
    17. Rochelle M. Edge & Michael T. Kiley & Jean-Philippe Laforte, 2010. "A comparison of forecast performance between Federal Reserve staff forecasts, simple reduced-form models, and a DSGE model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 720-754.
    18. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    19. Diebold, Francis X. & Rudebusch, Glenn D. & Borag[caron]an Aruoba, S., 2006. "The macroeconomy and the yield curve: a dynamic latent factor approach," Journal of Econometrics, Elsevier, vol. 131(1-2), pages 309-338.
    20. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    2. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    3. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    4. Chauvet, Marcelle & Potter, Simon, 2013. "Forecasting Output," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 141-194, Elsevier.
    5. Goulet Coulombe, Philippe & Leroux, Maxime & Stevanovic, Dalibor & Surprenant, Stéphane, 2021. "Macroeconomic data transformations matter," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1338-1354.
    6. Hännikäinen, Jari, 2017. "When does the yield curve contain predictive power? Evidence from a data-rich environment," International Journal of Forecasting, Elsevier, vol. 33(4), pages 1044-1064.
    7. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    8. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    9. Norman R. Swanson & Weiqi Xiong & Xiye Yang, 2020. "Predicting interest rates using shrinkage methods, real‐time diffusion indexes, and model combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 587-613, August.
    10. Henzel, Steffen R. & Mayr, Johannes, 2013. "The mechanics of VAR forecast pooling—A DSGE model based Monte Carlo study," The North American Journal of Economics and Finance, Elsevier, vol. 24(C), pages 1-24.
    11. Mu-Chun Wang, 2009. "Comparing the DSGE model with the factor model: an out-of-sample forecasting experiment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(2), pages 167-182.
    12. Maik H. Wolters, 2015. "Evaluating Point and Density Forecasts of DSGE Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(1), pages 74-96, January.
    13. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    14. Groen, Jan J.J. & Kapetanios, George, 2016. "Revisiting useful approaches to data-rich macroeconomic forecasting," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 221-239.
    15. Hauzenberger, Niko & Huber, Florian & Klieber, Karin, 2023. "Real-time inflation forecasting using non-linear dimension reduction techniques," International Journal of Forecasting, Elsevier, vol. 39(2), pages 901-921.
    16. Baltagi, Badi H., 2013. "Panel Data Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 995-1024, Elsevier.
    17. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    18. Johanna Posch & Fabio Rumler, 2015. "Semi‐Structural Forecasting of UK Inflation Based on the Hybrid New Keynesian Phillips Curve," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(2), pages 145-162, March.
    19. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    20. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2203.06540. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.