IDEAS home Printed from https://ideas.repec.org/p/fip/fednsr/161.html
   My bibliography  Save this paper

Modeling uncertainty: predictive accuracy as a proxy for predictive confidence

Author

Listed:
  • Robert W. Rich
  • Joseph Tracy

Abstract

This paper evaluates current strategies for the empirical modeling of forecast behavior. In particular, we focus on the reliability of using proxies from time series models of heteroskedasticity to describe changes in predictive confidence. We address this issue by examining the relationship between ex post forecast errors and ex ante measures of forecast uncertainty from data on inflation forecasts from the Survey of Professional Forecasters. The results provide little evidence of a strong link between observed heteroskedasticity in the consensus forecast errors and forecast uncertainty. Instead, the findings indicate a significant link between observed heteroskedasticity in the consensus forecast errors and forecast dispersion. We conclude that conventional model-based measures of uncertainty may be capturing not the degree of confidence that individuals attach to their forecasts but rather the degree of disagreement across individuals in their forecasts.

Suggested Citation

  • Robert W. Rich & Joseph Tracy, 2003. "Modeling uncertainty: predictive accuracy as a proxy for predictive confidence," Staff Reports 161, Federal Reserve Bank of New York.
  • Handle: RePEc:fip:fednsr:161
    as

    Download full text from publisher

    File URL: https://www.newyorkfed.org/medialibrary/media/research/staff_reports/sr161.html
    Download Restriction: no

    File URL: https://www.newyorkfed.org/medialibrary/media/research/staff_reports/sr161.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. Pennacchi, George G, 1991. "Identifying the Dynamics of Real Interest Rates and Inflation: Evidence Using Survey Data," Review of Financial Studies, Society for Financial Studies, vol. 4(1), pages 53-86.
    4. Cukierman, Alex & Wachtel, Paul, 1979. "Differential Inflationary Expectations and the Variability of the Rate of Inflation: Theory and Evidence," American Economic Review, American Economic Association, vol. 69(4), pages 595-609, September.
    5. Lucas, Robert Jr., 1972. "Expectations and the neutrality of money," Journal of Economic Theory, Elsevier, vol. 4(2), pages 103-124, April.
    6. Lahiri, Kajal & Teigland, Christie & Zaporowski, Mark, 1988. "Interest Rates and the Subjective Probability Distribution of Inflation Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 20(2), pages 233-248, May.
    7. Christopher D Carroll, 2001. "The Epidemiology of Macroeconomic Expectations," Economics Working Paper Archive 462, The Johns Hopkins University,Department of Economics.
    8. Dean Croushore, 1993. "Introducing: the survey of professional forecasters," Business Review, Federal Reserve Bank of Philadelphia, issue Nov, pages 3-15.
    9. Lloyd B. Thomas, 1999. "Survey Measures of Expected U.S. Inflation," Journal of Economic Perspectives, American Economic Association, vol. 13(4), pages 125-144, Fall.
    10. Francis X. Diebold & Anthony S. Tay & Kenneth F. Wallis, 1997. "Evaluating Density Forecasts of Inflation: The Survey of Professional Forecasters," NBER Working Papers 6228, National Bureau of Economic Research, Inc.
    11. Nicholas S. Souleles, 2001. "Consumer Sentiment: Its Rationality and Usefulness in Forecasting Expenditure - Evidence from the Michigan Micro Data," NBER Working Papers 8410, National Bureau of Economic Research, Inc.
    12. Batchelor, Roy & Dua, Pami, 1991. "Blue Chip Rationality Tests," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 23(4), pages 692-705, November.
    13. Victor Zarnowitz, 1984. "Business Cycles Analysis and Expectational Survey Data," NBER Working Papers 1378, National Bureau of Economic Research, Inc.
    14. Bollerslev, Tim & Chou, Ray Y. & Kroner, Kenneth F., 1992. "ARCH modeling in finance : A review of the theory and empirical evidence," Journal of Econometrics, Elsevier, vol. 52(1-2), pages 5-59.
    15. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    16. Bonham, Carl & Cohen, Richard, 1995. "Testing the Rationality of Price Forecasts: Comment," American Economic Review, American Economic Association, vol. 85(1), pages 284-289, March.
    17. Bomberger, William A, 1996. "Disagreement as a Measure of Uncertainty," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 28(3), pages 381-392, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Capistrán, Carlos, 2008. "Bias in Federal Reserve inflation forecasts: Is the Federal Reserve irrational or just cautious?," Journal of Monetary Economics, Elsevier, vol. 55(8), pages 1415-1427, November.
    2. Carlos Capistrán & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    3. Hartmann, Matthias & Herwartz, Helmut & Ulm, Maren, 2017. "A comparative assessment of alternative ex ante measures of inflation uncertainty," International Journal of Forecasting, Elsevier, vol. 33(1), pages 76-89.
    4. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    5. Stefania D'Amico & Athanasios Orphanides, 2008. "Uncertainty and disagreement in economic forecasting," Finance and Economics Discussion Series 2008-56, Board of Governors of the Federal Reserve System (U.S.).
    6. Jonas Dovern & Ulrich Fritsche, 2008. "Estimating fundamental cross-section dispersion from fixed event forecasts," Macroeconomics and Finance Series 200801, Hamburg University, Department Wirtschaft und Politik.
    7. Clements, Michael P., 2008. "Consensus and uncertainty: Using forecast probabilities of output declines," International Journal of Forecasting, Elsevier, vol. 24(1), pages 76-86.
    8. Hussain, Syed Mujahid, 2011. "Simultaneous monetary policy announcements and international stock markets response: An intraday analysis," Journal of Banking & Finance, Elsevier, vol. 35(3), pages 752-764, March.
    9. Van Nieuwerburgh, Stijn & Veldkamp, Laura, 2006. "Learning asymmetries in real business cycles," Journal of Monetary Economics, Elsevier, vol. 53(4), pages 753-772, May.
    10. Fushang Liu & Kajal Lahiri, 2006. "Modelling multi-period inflation uncertainty using a panel of density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(8), pages 1199-1219.
    11. Magnus Andersson & Lars Jul Overby & Szabolcs Sebestyén, 2009. "Which News Moves the Euro Area Bond Market?," German Economic Review, Verein für Socialpolitik, vol. 10, pages 1-31, February.
    12. Lahiri, Kajal & Liu, Fushang, 2005. "ARCH models for multi-period forecast uncertainty-a reality check using a panel of density forecasts," MPRA Paper 21693, University Library of Munich, Germany.
    13. Andersen, Torben G. & Bollerslev, Tim & Diebold, Francis X. & Vega, Clara, 2007. "Real-time price discovery in global stock, bond and foreign exchange markets," Journal of International Economics, Elsevier, vol. 73(2), pages 251-277, November.
    14. Clements, Michael P, 2006. "Internal consistency of survey respondents.forecasts : Evidence based on the Survey of Professional Forecasters," The Warwick Economics Research Paper Series (TWERPS) 772, University of Warwick, Department of Economics.

    More about this item

    Keywords

    Forecasting ; Time-series analysis ; Business cycles;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fednsr:161. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Amy Farber). General contact details of provider: http://edirc.repec.org/data/frbnyus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.