IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v35y2019i4p1564-1582.html
   My bibliography  Save this article

Gauging the uncertainty of the economic outlook using historical forecasting errors: The Federal Reserve’s approach

Author

Listed:
  • Reifschneider, David
  • Tulip, Peter

Abstract

The Federal Open Market Committee (FOMC) of the U.S. Federal Reserve regularly publishes participants’ qualitative assessments of forecast uncertainty, expressed relative to that seen on average in the past. The benchmarks used for these historical comparisons are the average root mean squared forecast errors (RMSEs) made by various private and government forecasters over the past twenty years. This paper documents how these benchmarks are constructed and discusses some of their properties. We draw several conclusions. First, if past performance is a reasonable guide to future accuracy, considerable uncertainty surrounds macroeconomic projections. Second, different forecasters have similar accuracy. Third, estimates of uncertainty about future real activity and interest rates are now considerably greater than prior to the financial crisis; in contrast, estimates of inflation accuracy have changed little. Finally, fan charts, constructed under certain assumptions and viewed in conjunction with the FOMC’s qualitative assessments, provide a reasonable approximation to future uncertainty.

Suggested Citation

  • Reifschneider, David & Tulip, Peter, 2019. "Gauging the uncertainty of the economic outlook using historical forecasting errors: The Federal Reserve’s approach," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1564-1582.
  • Handle: RePEc:eee:intfor:v:35:y:2019:i:4:p:1564-1582
    DOI: 10.1016/j.ijforecast.2018.07.016
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207018301432
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2018.07.016?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kajal Lahiri & Huaming Peng & Xuguang Simon Sheng, 2022. "Measuring Uncertainty of a Combined Forecast and Some Tests for Forecaster Heterogeneity," Advances in Econometrics, in: Essays in Honor of M. Hashem Pesaran: Prediction and Macro Modeling, volume 43, pages 29-50, Emerald Group Publishing Limited.
    2. Diebold, Francis X. & Schorfheide, Frank & Shin, Minchul, 2017. "Real-time forecast evaluation of DSGE models with stochastic volatility," Journal of Econometrics, Elsevier, vol. 201(2), pages 322-332.
    3. Fair, Ray C., 2014. "How might a central bank report uncertainty?," Economics Discussion Papers 2014-25, Kiel Institute for the World Economy (IfW Kiel).
    4. Christian Bauer & Sebastian Weber, 2016. "The Efficiency of Monetary Policy when Guiding Inflation Expectations," Research Papers in Economics 2016-14, University of Trier, Department of Economics.
    5. Flint Brayton & Thomas Laubach & David L. Reifschneider, 2014. "The FRB/US Model: A Tool for Macroeconomic Policy Analysis," FEDS Notes 2014-04-03, Board of Governors of the Federal Reserve System (U.S.).
    6. Christopher A. Sims, 2002. "The Role of Models and Probabilities in the Monetary Policy Process," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 33(2), pages 1-62.
    7. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2016. "Common Drifting Volatility in Large Bayesian VARs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 375-390, July.
    8. West, Kenneth D. & Cho, Dongchul, 1995. "The predictive ability of several models of exchange rate volatility," Journal of Econometrics, Elsevier, vol. 69(2), pages 367-391, October.
    9. Fair, Ray C, 1980. "Estimating the Expected Predictive Accuracy of Econometric Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(2), pages 355-378, June.
    10. West, Kenneth D. & Edison, Hali J. & Cho, Dongchul, 1993. "A utility-based comparison of some models of exchange rate volatility," Journal of International Economics, Elsevier, vol. 35(1-2), pages 23-45, August.
    11. Holston, Kathryn & Laubach, Thomas & Williams, John C., 2017. "Measuring the natural rate of interest: International trends and determinants," Journal of International Economics, Elsevier, vol. 108(S1), pages 59-75.
    12. Alejandro Justiniano & Giorgio E. Primiceri, 2008. "The Time-Varying Volatility of Macroeconomic Fluctuations," American Economic Review, American Economic Association, vol. 98(3), pages 604-641, June.
    13. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    14. Faust, Jon & Wright, Jonathan H., 2009. "Comparing Greenbook and Reduced Form Forecasts Using a Large Realtime Dataset," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 468-479.
    15. Peter Tulip, 2009. "Has the Economy Become More Predictable? Changes in Greenbook Forecast Accuracy," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(6), pages 1217-1231, September.
    16. Antonello D'Agostino & Luca Gambetti & Domenico Giannone, 2013. "Macroeconomic forecasting and structural change," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(1), pages 82-101, January.
    17. Todd E. Clark & Michael W. McCracken & Elmar Mertens, 2020. "Modeling Time-Varying Uncertainty of Multiple-Horizon Forecast Errors," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 17-33, March.
    18. Croushore Dean, 2010. "An Evaluation of Inflation Forecasts from Surveys Using Real-Time Data," The B.E. Journal of Macroeconomics, De Gruyter, vol. 10(1), pages 1-32, May.
    19. Fair, Ray C., 2014. "How might a central bank report uncertainty?," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 8, pages 1-22.
    20. Michael D. Bauer & Glenn D. Rudebusch, 2016. "Monetary Policy Expectations at the Zero Lower Bound," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 48(7), pages 1439-1465, October.
    21. David H. Romer & Christina D. Romer, 2000. "Federal Reserve Information and the Behavior of Interest Rates," American Economic Review, American Economic Association, vol. 90(3), pages 429-457, June.
    22. Peter Tulip & Stephanie Wallace, 2012. "Estimates of Uncertainty around the RBA's Forecasts," RBA Research Discussion Papers rdp2012-07, Reserve Bank of Australia.
    23. Malte Knüppel & Guido Schultefrankenfeld, 2017. "Interest rate assumptions and predictive accuracy of central bank forecasts," Empirical Economics, Springer, vol. 53(1), pages 195-215, August.
    24. Malte Knüppel & Guido Schultefrankenfeld, 2012. "How Informative Are Central Bank Assessments of Macroeconomic Risks?," International Journal of Central Banking, International Journal of Central Banking, vol. 8(3), pages 87-139, September.
    25. Lukas Vogel, 2007. "How do the OECD Growth Projections for the G7 Economies Perform?: A Post-Mortem," OECD Economics Department Working Papers 573, OECD Publishing.
    26. William T. Gavin & Rachel J. Mandal, 2001. "Forecasting inflation and growth: do private forecasts match those of policymakers?," Review, Federal Reserve Bank of St. Louis, vol. 83(May), pages 11-20.
    27. Kenneth F. Wallis, 1999. "Asymmetric density forecasts of inflation and the Bank of England's fan chart," National Institute Economic Review, National Institute of Economic and Social Research, vol. 167(1), pages 106-112, January.
    28. Knüppel, Malte & Vladu, Andreea L., 2016. "Approximating fixed-horizon forecasts using fixed-event forecasts," Discussion Papers 28/2016, Deutsche Bundesbank.
    29. Campbell, Sean D., 2007. "Macroeconomic Volatility, Predictability, and Uncertainty in the Great Moderation: Evidence From the Survey of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 191-200, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kajal Lahiri & Huaming Peng & Xuguang Simon Sheng, 2022. "Measuring Uncertainty of a Combined Forecast and Some Tests for Forecaster Heterogeneity," Advances in Econometrics, in: Essays in Honor of M. Hashem Pesaran: Prediction and Macro Modeling, volume 43, pages 29-50, Emerald Group Publishing Limited.
    2. Carola Conces Binder & Rodrigo Sekkel, 2023. "Central Bank Forecasting: A Survey," Staff Working Papers 23-18, Bank of Canada.
    3. Alberto Caruso & Laura Coroneo, 2023. "Does Real‐Time Macroeconomic Information Help to Predict Interest Rates?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 55(8), pages 2027-2059, December.
    4. Mary C. Daly, 2023. "What the Moment Demands," FRBSF Economic Letter, Federal Reserve Bank of San Francisco, vol. 2023(30), pages 1-6, November.
    5. Bundick, Brent & Herriford, Trenton & Smith, A. Lee, 2024. "The Term Structure of Monetary Policy Uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 160(C).
    6. Adams, Patrick A. & Adrian, Tobias & Boyarchenko, Nina & Giannone, Domenico, 2021. "Forecasting macroeconomic risks," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1173-1191.
    7. Martinez, Andrew & Schibuola, Alex, 2021. "The Expectations Gap: An Alternative Measure of Economic Slack," Working Papers 11284, George Mason University, Mercatus Center.
    8. Tsuchiya, Yoichi, 2023. "Assessing the World Bank’s growth forecasts," Economic Analysis and Policy, Elsevier, vol. 77(C), pages 64-84.
    9. Galvão, Ana Beatriz & Garratt, Anthony & Mitchell, James, 2021. "Does judgment improve macroeconomic density forecasts?," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1247-1260.
    10. G. Kontogeorgos & K. Lambrias, 2022. "Evaluating the Eurosystem/ECB staff macroeconomic projections: The first 20 years," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 213-229, March.
    11. Knotek, Edward S. & Zaman, Saeed, 2023. "Real-time density nowcasts of US inflation: A model combination approach," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1736-1760.
    12. Marián Vávra, 2020. "Assessing distributional properties of forecast errors for fan-chart modelling," Empirical Economics, Springer, vol. 59(6), pages 2841-2858, December.
    13. Kevin L Kliesen, 2023. "A Comparison of Fed "Tightening" Episodes since the 1980s," International Journal of Central Banking, International Journal of Central Banking, vol. 19(3), pages 423-450, August.
    14. Fabian Kruger & Hendrik Plett, 2022. "Prediction intervals for economic fixed-event forecasts," Papers 2210.13562, arXiv.org, revised Mar 2024.
    15. Tsuchiya, Yoichi, 2022. "Evaluating the European Central Bank’s uncertainty forecasts," Economic Analysis and Policy, Elsevier, vol. 73(C), pages 321-330.
    16. Philip Hans Franses, 2021. "Modeling Judgment in Macroeconomic Forecasts," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 19(1), pages 401-417, December.
    17. James Mitchell & Martin Weale, 2023. "Censored density forecasts: Production and evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 714-734, August.
    18. Timo Dimitriadis & Andrew J. Patton & Patrick W. Schmidt, 2019. "Testing Forecast Rationality for Measures of Central Tendency," Papers 1910.12545, arXiv.org, revised Jun 2023.
    19. Andrew C. Chang & Trace J. Levinson, 2023. "Raiders of the lost high‐frequency forecasts: New data and evidence on the efficiency of the Fed's forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(1), pages 88-104, January.
    20. Todd E. Clark & Gergely Ganics & Elmar Mertens, 2022. "Constructing Fan Charts from the Ragged Edge of SPF Forecasts," Working Papers 22-36, Federal Reserve Bank of Cleveland.
    21. Ding, Yibing & Liu, Ziyu & Liu, Dayu, 2022. "Structural news shock, financial market uncertainty and China's business fluctuations," Pacific-Basin Finance Journal, Elsevier, vol. 76(C).
    22. Oscar Claveria, 2021. "Disagreement on expectations: firms versus consumers," SN Business & Economics, Springer, vol. 1(12), pages 1-23, December.
    23. Gregory R. Duffee, 2023. "Macroeconomic News in Asset Pricing and Reality," Journal of Finance, American Finance Association, vol. 78(3), pages 1499-1543, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David L. Reifschneider & Peter Tulip, 2017. "Gauging the Uncertainty of the Economic Outlook Using Historical Forecasting Errors : The Federal Reserve's Approach," Finance and Economics Discussion Series 2017-020, Board of Governors of the Federal Reserve System (U.S.).
    2. Todd E. Clark & Michael W. McCracken & Elmar Mertens, 2020. "Modeling Time-Varying Uncertainty of Multiple-Horizon Forecast Errors," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 17-33, March.
    3. Paul Hubert, 2015. "Revisiting the Greenbook’s relative forecasting performance," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(1), pages 151-179.
    4. Liu, Dandan & Smith, Julie K., 2014. "Inflation forecasts and core inflation measures: Where is the information on future inflation?," The Quarterly Review of Economics and Finance, Elsevier, vol. 54(1), pages 133-137.
    5. repec:spo:wpmain:info:hdl:2441/3pot7260lh88lrfhrhvs85lh2f is not listed on IDEAS
    6. Todd E. Clark & Francesco Ravazzolo, 2012. "The macroeconomic forecasting performance of autoregressive models with alternative specifications of time-varying volatility," Working Paper 2012/09, Norges Bank.
    7. repec:hal:spmain:info:hdl:2441/3pot7260lh88lrfhrhvs85lh2f is not listed on IDEAS
    8. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2015. "Realtime nowcasting with a Bayesian mixed frequency model with stochastic volatility," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 837-862, October.
    9. Knüppel, Malte & Schultefrankenfeld, Guido, 2019. "Assessing the uncertainty in central banks’ inflation outlooks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1748-1769.
    10. Faust, Jon & Wright, Jonathan H., 2013. "Forecasting Inflation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 2-56, Elsevier.
    11. Paul Hubert, 2009. "An Empirical Review of Federal Reserve’s Informational Advantage," Documents de Travail de l'OFCE 2009-03, Observatoire Francais des Conjonctures Economiques (OFCE).
    12. repec:hal:spmain:info:hdl:2441/35kgubh40v9gfpnuruelqjnptb is not listed on IDEAS
    13. David L. Reifschneider & Peter Tulip, 2007. "Gauging the uncertainty of the economic outlook from historical forecasting errors," Finance and Economics Discussion Series 2007-60, Board of Governors of the Federal Reserve System (U.S.).
    14. Julien Champagne & Guillaume Poulin‐Bellisle & Rodrigo Sekkel, 2020. "Introducing the Bank of Canada staff economic projections database," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(1), pages 114-129, January.
    15. Fabian Krüger & Todd E. Clark & Francesco Ravazzolo, 2017. "Using Entropic Tilting to Combine BVAR Forecasts With External Nowcasts," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(3), pages 470-485, July.
    16. Peter Tulip & Stephanie Wallace, 2012. "Estimates of Uncertainty around the RBA's Forecasts," RBA Research Discussion Papers rdp2012-07, Reserve Bank of Australia.
    17. Berge, Travis J. & Chang, Andrew C. & Sinha, Nitish R., 2019. "Evaluating the conditionality of judgmental forecasts," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1627-1635.
    18. Arai, Natsuki, 2014. "Using forecast evaluation to improve the accuracy of the Greenbook forecast," International Journal of Forecasting, Elsevier, vol. 30(1), pages 12-19.
    19. Matei Demetrescu & Christoph Hanck & Robinson Kruse‐Becher, 2022. "Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 1010-1030, August.
    20. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    21. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2016. "Common Drifting Volatility in Large Bayesian VARs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 375-390, July.
    22. Pablo Guerróon‐Quintana & Molin Zhong, 2023. "Macroeconomic forecasting in times of crises," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(3), pages 295-320, April.
    23. Gamber, Edward N. & Smith, Julie K., 2009. "Are the Fed's inflation forecasts still superior to the private sector's?," Journal of Macroeconomics, Elsevier, vol. 31(2), pages 240-251, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:35:y:2019:i:4:p:1564-1582. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.