IDEAS home Printed from https://ideas.repec.org/a/ebl/ecbull/eb-12-00339.html
   My bibliography  Save this article

A new approach for evaluating economic forecasts

Author

Listed:
  • Tara M. Sinclair

    (The George Washington University)

  • H. O. Stekler

    (The George Washington University)

  • Warren Carnow

    (The George Washington University)

Abstract

This paper presents a recently developed approach for evaluating economic forecasts. Previously, univariate methods were used to evaluate the forecasts of individual variables. However, many macroeconomic variables are forecast at the same time to describe the state of the economy. It is, therefore, appropriate to use a multivariate methodology in evaluating these forecasts. Our approach uses VARs and distance measures. It is applied to the Survey of Professional Forecasters (SPF). Our contributions are the application of the methodology for evaluating multivariate forecasts to the SPF, measuring accuracy, and testing for bias within this framework. We also consider whether there are forecasting performance asymmetries over the business cycle.

Suggested Citation

  • Tara M. Sinclair & H. O. Stekler & Warren Carnow, 2012. "A new approach for evaluating economic forecasts," Economics Bulletin, AccessEcon, vol. 32(3), pages 2332-2342.
    • Handle: RePEc:ebl:ecbull:eb-12-00339
    as

    Download full text from publisher

    File URL: http://www.accessecon.com/Pubs/EB/2012/Volume32/EB-12-V32-I3-P226.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, "undated". "Evaluating Density Forecasts," CARESS Working Papres 97-18, University of Pennsylvania Center for Analytic Research and Economics in the Social Sciences.
    2. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    3. David H. Romer & Christina D. Romer, 2000. "Federal Reserve Information and the Behavior of Interest Rates," American Economic Review, American Economic Association, vol. 90(3), pages 429-457, June.
    4. Holden, K & Peel, D A, 1990. "On Testing for Unbiasedness and Efficiency of Forecasts," The Manchester School of Economic & Social Studies, University of Manchester, vol. 58(2), pages 120-127, June.
    5. Sinclair, Tara M. & Joutz, Fred & Stekler, H.O., 2010. "Can the Fed predict the state of the economy?," Economics Letters, Elsevier, vol. 108(1), pages 28-32, July.
    6. Christopher A. Sims, 2002. "The Role of Models and Probabilities in the Monetary Policy Process," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 33(2), pages 1-62.
    7. Tara Sinclair & H. O. Stekler & L. Kitzinger, 2010. "Directional forecasts of GDP and inflation: a joint evaluation with an application to Federal Reserve predictions," Applied Economics, Taylor & Francis Journals, vol. 42(18), pages 2289-2297.
    8. Sinclair, Tara M. & Stekler, H.O. & Carnow, Warren, 2015. "Evaluating a vector of the Fed’s forecasts," International Journal of Forecasting, Elsevier, vol. 31(1), pages 157-164.
    9. Engle, Robert F. & White (the late), Halbert (ed.), 1999. "Cointegration, Causality, and Forecasting: Festschrift in Honour of Clive W. J. Granger," OUP Catalogue, Oxford University Press, number 9780198296836, Decembrie.
    10. Sinclair, Tara M. & Gamber, Edward N. & Stekler, Herman & Reid, Elizabeth, 2012. "Jointly evaluating the Federal Reserve’s forecasts of GDP growth and inflation," International Journal of Forecasting, Elsevier, vol. 28(2), pages 309-314.
    11. Fred Joutz & Michael P. Clements & Herman O. Stekler, 2007. "An evaluation of the forecasts of the federal reserve: a pooled approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(1), pages 121-136.
    12. Harvey, David I. & Newbold, Paul, 2003. "The non-normality of some macroeconomic forecast errors," International Journal of Forecasting, Elsevier, vol. 19(4), pages 635-653.
    13. Jordà, Òscar & Knüppel, Malte & Marcellino, Massimiliano, 2013. "Empirical simultaneous prediction regions for path-forecasts," International Journal of Forecasting, Elsevier, vol. 29(3), pages 456-468.
    14. Baghestani, Hamid, 2006. "An evaluation of the professional forecasts of U.S. long-term interest rates," Review of Financial Economics, Elsevier, vol. 15(2), pages 177-191.
    15. Joutz, Fred & Stekler, H. O., 2000. "An evaluation of the predictions of the Federal Reserve," International Journal of Forecasting, Elsevier, vol. 16(1), pages 17-38.
    16. Lahiri, Kajal & Teigland, Christie, 1987. "On the normality of probability distributions of inflation and GNP forecasts," International Journal of Forecasting, Elsevier, vol. 3(2), pages 269-279.
    17. Groen, Jan J.J. & Kapetanios, George & Price, Simon, 2009. "A real time evaluation of Bank of England forecasts of inflation and growth," International Journal of Forecasting, Elsevier, vol. 25(1), pages 74-80.
    18. Baghestani, Hamid, 1994. "Evaluating multiperiod survey forecasts of real net exports," Economics Letters, Elsevier, vol. 44(3), pages 267-272.
    19. Michael Clements, 2006. "Evaluating the survey of professional forecasters probability distributions of expected inflation based on derived event probability forecasts," Empirical Economics, Springer, vol. 31(1), pages 49-64, March.
    20. David L. Reifschneider & Peter Tulip, 2007. "Gauging the uncertainty of the economic outlook from historical forecasting errors," Finance and Economics Discussion Series 2007-60, Board of Governors of the Federal Reserve System (U.S.).
    21. Chanont Banternghansa & Michael W. McCracken, 2009. "Forecast disagreement among FOMC members," Working Papers 2009-059, Federal Reserve Bank of St. Louis.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hendry, David F. & Martinez, Andrew B., 2017. "Evaluating multi-step system forecasts with relatively few forecast-error observations," International Journal of Forecasting, Elsevier, vol. 33(2), pages 359-372.
    2. Kim, Jong Min & Jun, Mina & Kim, Chung K., 2018. "The Effects of Culture on Consumers' Consumption and Generation of Online Reviews," Journal of Interactive Marketing, Elsevier, vol. 43(C), pages 134-150.
    3. Glas, Alexander & Heinisch, Katja, 2021. "Conditional macroeconomic forecasts: Disagreement, revisions and forecast errors," IWH Discussion Papers 7/2021, Halle Institute for Economic Research (IWH).
    4. Ines Fortin & Sebastian P. Koch & Klaus Weyerstrass, 2020. "Evaluation of economic forecasts for Austria," Empirical Economics, Springer, vol. 58(1), pages 107-137, January.
    5. Ericsson, Neil R., 2017. "How biased are U.S. government forecasts of the federal debt?," International Journal of Forecasting, Elsevier, vol. 33(2), pages 543-559.
    6. Sinclair, Tara M. & Stekler, H.O. & Carnow, Warren, 2015. "Evaluating a vector of the Fed’s forecasts," International Journal of Forecasting, Elsevier, vol. 31(1), pages 157-164.
    7. Stekler, Herman & Symington, Hilary, 2016. "Evaluating qualitative forecasts: The FOMC minutes, 2006–2010," International Journal of Forecasting, Elsevier, vol. 32(2), pages 559-570.
    8. An, Zidong & Ball, Laurence & Jalles, Joao & Loungani, Prakash, 2019. "Do IMF forecasts respect Okun’s law? Evidence for advanced and developing economies," International Journal of Forecasting, Elsevier, vol. 35(3), pages 1131-1142.
    9. Ball, Laurence & Jalles, João Tovar & Loungani, Prakash, 2015. "Do forecasters believe in Okun’s Law? An assessment of unemployment and output forecasts," International Journal of Forecasting, Elsevier, vol. 31(1), pages 176-184.
    10. Tim Köhler & Jörg Döpke, 2023. "Will the last be the first? Ranking German macroeconomic forecasters based on different criteria," Empirical Economics, Springer, vol. 64(2), pages 797-832, February.
    11. Håvard Hungnes, 2020. "Equal predictability test for multi-step-ahead system forecasts invariant to linear transformations," Discussion Papers 931, Statistics Norway, Research Department.
    12. Sergey V. Smirnov & Daria A. Avdeeva, 2016. "Wishful Bias in Predicting Us Recessions: Indirect Evidence," HSE Working papers WP BRP 135/EC/2016, National Research University Higher School of Economics.
    13. Döhrn, Roland, 2015. "Der Prognostiker des Jahres: Ein Zufallsergebnis? Möglichkeiten einer mehrdimensionalen Evaluierung von Konjunkturprognosen," IBES Diskussionsbeiträge 208, University of Duisburg-Essen, Institute of Business and Economic Studie (IBES).
    14. Herman O. Stekler & Hilary Symington, 2014. "How Did The Fomc View The Great Recession As It Was Happening?: Evaluating The Minutes From Fomc Meetings, 2006-2010," Working Papers 2014-005, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    15. Behrens, Christoph, 2019. "Evaluating the Joint Efficiency of German Trade Forecasts. A nonparametric multivariate approach," Working Papers 9, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    16. Sergey V. Smirnov, 2014. "Predicting US Recessions: Does a Wishful Bias Exist?," HSE Working papers WP BRP 77/EC/2014, National Research University Higher School of Economics.
    17. Arai, Natsuki, 2020. "Investigating the inefficiency of the CBO’s budgetary projections," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1290-1300.
    18. Eicher, Theo S. & Rollinson, Yuan Gao, 2023. "The accuracy of IMF crises nowcasts," International Journal of Forecasting, Elsevier, vol. 39(1), pages 431-449.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sinclair, Tara M. & Stekler, H.O. & Carnow, Warren, 2015. "Evaluating a vector of the Fed’s forecasts," International Journal of Forecasting, Elsevier, vol. 31(1), pages 157-164.
    2. Messina, Jeffrey D. & Sinclair, Tara M. & Stekler, Herman, 2015. "What can we learn from revisions to the Greenbook forecasts?," Journal of Macroeconomics, Elsevier, vol. 45(C), pages 54-62.
    3. Tara M. Sinclair & H.O. Stekler, 2011. "Differences in Early GDP Component Estimates Between Recession and Expansion," Working Papers 2011-05, The George Washington University, Institute for International Economic Policy.
    4. Sinclair, Tara M. & Stekler, H.O., 2013. "Examining the quality of early GDP component estimates," International Journal of Forecasting, Elsevier, vol. 29(4), pages 736-750.
    5. Henning Fischer & Marta García-Bárzana & Peter Tillmann & Peter Winker, 2014. "Evaluating FOMC forecast ranges: an interval data approach," Empirical Economics, Springer, vol. 47(1), pages 365-388, August.
    6. Carola Conces Binder & Rodrigo Sekkel, 2023. "Central Bank Forecasting: A Survey," Staff Working Papers 23-18, Bank of Canada.
    7. Paul Hubert, 2015. "Revisiting the Greenbook’s relative forecasting performance," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(1), pages 151-179.
    8. Sinclair, Tara M. & Gamber, Edward N. & Stekler, Herman & Reid, Elizabeth, 2012. "Jointly evaluating the Federal Reserve’s forecasts of GDP growth and inflation," International Journal of Forecasting, Elsevier, vol. 28(2), pages 309-314.
    9. repec:hal:spmain:info:hdl:2441/35kgubh40v9gfpnuruelqjnptb is not listed on IDEAS
    10. Tara M. Sinclair & Hans Christian Müller-Dröge & Herman Stekler, 2014. "Evaluating Forecasts of a Vector of Variables: A German Forecasting Competition," Working Papers 2014-17, The George Washington University, Institute for International Economic Policy.
    11. Ericsson, Neil R., 2016. "Eliciting GDP forecasts from the FOMC’s minutes around the financial crisis," International Journal of Forecasting, Elsevier, vol. 32(2), pages 571-583.
    12. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    13. Michael T. Belongia & Peter N. Ireland, 2018. "Monetary Policy Lessons from the Greenbook," Boston College Working Papers in Economics 955, Boston College Department of Economics.
    14. Pao‐Lin Tien & Tara M. Sinclair & Edward N. Gamber, 2021. "Do Fed Forecast Errors Matter?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(3), pages 686-712, June.
    15. Baghestani, Hamid & AbuAl-Foul, Bassam M., 2017. "Comparing Federal Reserve, Blue Chip, and time series forecasts of US output growth," Journal of Economics and Business, Elsevier, vol. 89(C), pages 47-56.
    16. Xie, Zixiong & Hsu, Shih-Hsun, 2016. "Time varying biases and the state of the economy," International Journal of Forecasting, Elsevier, vol. 32(3), pages 716-725.
    17. Fred Joutz & Michael P. Clements & Herman O. Stekler, 2007. "An evaluation of the forecasts of the federal reserve: a pooled approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(1), pages 121-136.
    18. Berge, Travis J. & Chang, Andrew C. & Sinha, Nitish R., 2019. "Evaluating the conditionality of judgmental forecasts," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1627-1635.
    19. Julien Champagne & Guillaume Poulin-Bellisle & Rodrigo Sekkel, 2018. "Evaluating the Bank of Canada Staff Economic Projections Using a New Database of Real-Time Data and Forecasts," Staff Working Papers 18-52, Bank of Canada.
    20. Tara M. Sinclair & Fred Joutz & Herman O. Stekler, 2008. "Are 'unbiased' forecasts really unbiased? Another look at the Fed forecasts," Working Papers 2008-010, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    21. Chang, Andrew C. & Hanson, Tyler J., 2016. "The accuracy of forecasts prepared for the Federal Open Market Committee," Journal of Economics and Business, Elsevier, vol. 83(C), pages 23-43.

    More about this item

    Keywords

    Forecast Evaluation; Survey of Professional Forecasters; Business Cycle; Mahalanobis Distance;
    All these keywords.

    JEL classification:

    • C5 - Mathematical and Quantitative Methods - - Econometric Modeling
    • E3 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ebl:ecbull:eb-12-00339. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: John P. Conley (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.