IDEAS home Printed from https://ideas.repec.org/p/fip/fednsr/161.html
   My bibliography  Save this paper

Modeling uncertainty: predictive accuracy as a proxy for predictive confidence

Author

Listed:
  • Robert W. Rich
  • Joseph Tracy

Abstract

This paper evaluates current strategies for the empirical modeling of forecast behavior. In particular, we focus on the reliability of using proxies from time series models of heteroskedasticity to describe changes in predictive confidence. We address this issue by examining the relationship between ex post forecast errors and ex ante measures of forecast uncertainty from data on inflation forecasts from the Survey of Professional Forecasters. The results provide little evidence of a strong link between observed heteroskedasticity in the consensus forecast errors and forecast uncertainty. Instead, the findings indicate a significant link between observed heteroskedasticity in the consensus forecast errors and forecast dispersion. We conclude that conventional model-based measures of uncertainty may be capturing not the degree of confidence that individuals attach to their forecasts but rather the degree of disagreement across individuals in their forecasts.

Suggested Citation

  • Robert W. Rich & Joseph Tracy, 2003. "Modeling uncertainty: predictive accuracy as a proxy for predictive confidence," Staff Reports 161, Federal Reserve Bank of New York.
  • Handle: RePEc:fip:fednsr:161
    as

    Download full text from publisher

    File URL: https://www.newyorkfed.org/medialibrary/media/research/staff_reports/sr161.html
    Download Restriction: no

    File URL: https://www.newyorkfed.org/medialibrary/media/research/staff_reports/sr161.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. Cukierman, Alex & Wachtel, Paul, 1979. "Differential Inflationary Expectations and the Variability of the Rate of Inflation: Theory and Evidence," American Economic Review, American Economic Association, vol. 69(4), pages 595-609, September.
    4. Lahiri, Kajal & Teigland, Christie & Zaporowski, Mark, 1988. "Interest Rates and the Subjective Probability Distribution of Inflation Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 20(2), pages 233-248, May.
    5. Francis X. Diebold & Anthony S. Tay & Kenneth F. Wallis, 1997. "Evaluating Density Forecasts of Inflation: The Survey of Professional Forecasters," NBER Working Papers 6228, National Bureau of Economic Research, Inc.
    6. Nicholas S. Souleles, 2001. "Consumer Sentiment: Its Rationality and Usefulness in Forecasting Expenditure - Evidence from the Michigan Micro Data," NBER Working Papers 8410, National Bureau of Economic Research, Inc.
    7. Batchelor, Roy & Dua, Pami, 1991. "Blue Chip Rationality Tests," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 23(4), pages 692-705, November.
    8. Victor Zarnowitz, 1984. "Business Cycles Analysis and Expectational Survey Data," NBER Working Papers 1378, National Bureau of Economic Research, Inc.
    9. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    10. Bomberger, William A, 1996. "Disagreement as a Measure of Uncertainty," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 28(3), pages 381-392, August.
    11. Pennacchi, George G, 1991. "Identifying the Dynamics of Real Interest Rates and Inflation: Evidence Using Survey Data," Review of Financial Studies, Society for Financial Studies, vol. 4(1), pages 53-86.
    12. Lucas, Robert Jr., 1972. "Expectations and the neutrality of money," Journal of Economic Theory, Elsevier, vol. 4(2), pages 103-124, April.
    13. Christopher D Carroll, 2001. "The Epidemiology of Macroeconomic Expectations," Economics Working Paper Archive 462, The Johns Hopkins University,Department of Economics.
    14. Dean Croushore, 1993. "Introducing: the survey of professional forecasters," Business Review, Federal Reserve Bank of Philadelphia, issue Nov, pages 3-15.
    15. Lloyd B. Thomas, 1999. "Survey Measures of Expected U.S. Inflation," Journal of Economic Perspectives, American Economic Association, vol. 13(4), pages 125-144, Fall.
    16. Bollerslev, Tim & Chou, Ray Y. & Kroner, Kenneth F., 1992. "ARCH modeling in finance : A review of the theory and empirical evidence," Journal of Econometrics, Elsevier, vol. 52(1-2), pages 5-59.
    17. Bonham, Carl & Cohen, Richard, 1995. "Testing the Rationality of Price Forecasts: Comment," American Economic Review, American Economic Association, vol. 85(1), pages 284-289, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hartmann, Matthias & Herwartz, Helmut & Ulm, Maren, 2017. "A comparative assessment of alternative ex ante measures of inflation uncertainty," International Journal of Forecasting, Elsevier, vol. 33(1), pages 76-89.
    2. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    3. Jonas Dovern & Ulrich Fritsche, 2008. "Estimating Fundamental Cross-Section Dispersion from Fixed Event Forecasts," Discussion Papers of DIW Berlin 787, DIW Berlin, German Institute for Economic Research.
    4. Hussain, Syed Mujahid, 2011. "Simultaneous monetary policy announcements and international stock markets response: An intraday analysis," Journal of Banking & Finance, Elsevier, vol. 35(3), pages 752-764, March.
    5. Van Nieuwerburgh, Stijn & Veldkamp, Laura, 2006. "Learning asymmetries in real business cycles," Journal of Monetary Economics, Elsevier, vol. 53(4), pages 753-772, May.
    6. Lahiri, Kajal & Liu, Fushang, 2005. "ARCH models for multi-period forecast uncertainty-a reality check using a panel of density forecasts," MPRA Paper 21693, University Library of Munich, Germany.
    7. Capistrán, Carlos, 2008. "Bias in Federal Reserve inflation forecasts: Is the Federal Reserve irrational or just cautious?," Journal of Monetary Economics, Elsevier, vol. 55(8), pages 1415-1427, November.
    8. Andersen, Torben G. & Bollerslev, Tim & Diebold, Francis X. & Vega, Clara, 2007. "Real-time price discovery in global stock, bond and foreign exchange markets," Journal of International Economics, Elsevier, vol. 73(2), pages 251-277, November.
    9. Clements, Michael P., 2006. "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    10. Fushang Liu & Kajal Lahiri, 2006. "Modelling multi-period inflation uncertainty using a panel of density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(8), pages 1199-1219.
    11. Carlos Capistr¡N & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    12. Stefania D'Amico & Athanasios Orphanides, 2008. "Uncertainty and disagreement in economic forecasting," Finance and Economics Discussion Series 2008-56, Board of Governors of the Federal Reserve System (U.S.).
    13. Carlos Capistrán & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2‐3), pages 365-396, March.
    14. Clements, Michael P., 2008. "Consensus and uncertainty: Using forecast probabilities of output declines," International Journal of Forecasting, Elsevier, vol. 24(1), pages 76-86.
    15. Andersson, Magnus & Hansen, Lars Jul & Sebestyén, Szabolcs, 2006. "Which news moves the euro area bond market?," Working Paper Series 631, European Central Bank.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lahiri, Kajal & Liu, Fushang, 2005. "ARCH models for multi-period forecast uncertainty-a reality check using a panel of density forecasts," MPRA Paper 21693, University Library of Munich, Germany.
    2. Ball, Laurence & Croushore, Dean, 2003. "Expectations and the Effects of Monetary Policy," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 35(4), pages 473-484, August.
    3. Clements, Michael P., 2006. "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    4. N. Gregory Mankiw & Ricardo Reis & Justin Wolfers, 2004. "Disagreement about Inflation Expectations," NBER Chapters, in: NBER Macroeconomics Annual 2003, Volume 18, pages 209-270, National Bureau of Economic Research, Inc.
    5. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    6. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    7. Clements, Michael P., 2008. "Consensus and uncertainty: Using forecast probabilities of output declines," International Journal of Forecasting, Elsevier, vol. 24(1), pages 76-86.
    8. Davies, Antony, 2006. "A framework for decomposing shocks and measuring volatilities derived from multi-dimensional panel data of survey forecasts," International Journal of Forecasting, Elsevier, vol. 22(2), pages 373-393.
    9. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    10. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    11. repec:lan:wpaper:470 is not listed on IDEAS
    12. Bonham, Carl S & Cohen, Richard H, 2001. "To Aggregate, Pool, or Neither: Testing the Rational-Expectations Hypothesis Using Survey Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 278-291, July.
    13. Clements, Michael P., 2008. "Rounding of probability forecasts : The SPF forecast probabilities of negative output growth," The Warwick Economics Research Paper Series (TWERPS) 869, University of Warwick, Department of Economics.
    14. Jordi Pons-Novell, 2003. "Strategic bias, herding behaviour and economic forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 22(1), pages 67-77.
    15. Jean Sepulveda-Umanzor, 2004. "The Relation Between Macroeconomic Uncertainty And The Expected Performance Of the Economy," Econometric Society 2004 Latin American Meetings 304, Econometric Society.
    16. Christian Grimme & Steffen Henzel & Elisabeth Wieland, 2014. "Inflation uncertainty revisited: a proposal for robust measurement," Empirical Economics, Springer, vol. 47(4), pages 1497-1523, December.
    17. repec:lan:wpaper:425 is not listed on IDEAS
    18. repec:lan:wpaper:539557 is not listed on IDEAS
    19. repec:lan:wpaper:413 is not listed on IDEAS
    20. Hugo Benítez-Silva & Debra Dwyer & Wayne-Roy Gayle & Thomas Muench, 2008. "Expectations in micro data: rationality revisited," Empirical Economics, Springer, vol. 34(2), pages 381-416, March.
    21. Wändi Bruine de Bruin & Michael F. Bryan & Simon M. Potter & Giorgio Topa & Wilbert Van der Klaauw, 2008. "Rethinking the measurement of household inflation expectations: preliminary findings," Staff Reports 359, Federal Reserve Bank of New York.
    22. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    23. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    24. Silverio Foresi & Alessandro Penati & George Pennacchi, 1997. "Estimating the cost of U.S. indexed bonds," Working Papers (Old Series) 9701, Federal Reserve Bank of Cleveland.

    More about this item

    Keywords

    Forecasting; Time-series analysis; Business cycles;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fednsr:161. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/frbnyus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/frbnyus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.