Advanced Search
MyIDEAS: Login

The Calibration of Probabilistic Economic Forecasts

Contents:

Author Info

  • John Galbraith

    ()

  • Simon van Norden

    ()

Abstract

A probabilistic forecast is the estimated probability with which a future event will satisfy a specified criterion. One interesting feature of such forecasts is their calibration, or the match between predicted probabilities and actual outcome probabilities. Calibration has been evaluated in the past by grouping probability forecasts into discrete categories. Here we show that we can do so without discrete groupings; the kernel estimators that we use produce efficiency gains and smooth estimated curves relating predicted and actual probabilities. We use such estimates to evaluate the empirical evidence on calibration error in a number of economic applications including recession and inflation prediction, using both forecasts made and stored in real time and pseudoforecasts made using the data vintage available at the forecast date. We evaluate outcomes using both first-release outcome measures as well as later, thoroughly-revised data. We find strong evidence of incorrect calibration in professional forecasts of recessions and inflation. We also present evidence of asymmetries in the performance of inflation forecasts based on real-time output gaps. Une prévision probabiliste représente la probabilité qu’un événement futur satisfasse une condition donnée. Un des aspects intéressants de ces prévisions est leur calibration, c’est-à-dire l’appariement entre les probabilités prédites et les probabilités réalisées. Dans le passé, la calibration a été évaluée en regroupant des probabilités de prévisions en catégories distinctes. Nous proposons d’utiliser des estimateurs à noyaux, qui sont plus efficaces et qui estiment une relation lisse entre les probabilités prédites et réalisées. Nous nous servons de ces estimations pour évaluer l’importance empirique des erreurs de calibration dans plusieurs pratiques économiques, telles que la prévision de récessions et de l’inflation. Pour ce faire, nous utilisons des prévisions historiques, ainsi que des pseudoprévisions effectuées à l’aide de données telles qu’elles étaient au moment de la prévision. Nous analysons les résultats en utilisant autant des estimations préliminaires que des estimations tardives, ces dernières incorporant parfois des révisions importantes. Nous trouvons une forte évidence empirique d’une calibration erronée des prévisions professionnelles de récession et d’inflation. Nous présentons aussi une évidence d’asymétries dans la performance des prévisions d’inflation basées sur des estimations des écarts de la production en temps réel.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.cirano.qc.ca/pdf/publication/2008s-28.pdf
Download Restriction: no

Bibliographic Info

Paper provided by CIRANO in its series CIRANO Working Papers with number 2008s-28.

as in new window
Length:
Date of creation: 01 Nov 2008
Date of revision:
Handle: RePEc:cir:cirwor:2008s-28

Contact details of provider:
Postal: 2020 rue University, 25e étage, Montréal, Quéc, H3A 2A5
Phone: (514) 985-4000
Fax: (514) 985-4039
Email:
Web page: http://www.cirano.qc.ca/
More information through EDIRC

Related research

Keywords: calibration; probability forecast; real-time data; inflation; recession; calibration; probabilités de prévisions; données « en temps réel »; inflation; récession;

Other versions of this item:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Athanasios Orphanides & Simon Van_Norden, 2000. "The Reliability of Output Gap Estimates in Real Time," Econometric Society World Congress 2000 Contributed Papers 0768, Econometric Society.
  2. Watson, Mark W., 1986. "Univariate detrending methods with stochastic trends," Journal of Monetary Economics, Elsevier, vol. 18(1), pages 49-75, July.
  3. Athanasios Orphanides & Simon van Norden, 2004. "The reliability of inflation forecasts based on output gap estimates in real time," Finance and Economics Discussion Series 2004-68, Board of Governors of the Federal Reserve System (U.S.).
  4. Casillas-Olvera, Gabriel & Bessler, David A., 2006. "Probability forecasting and central bank accountability," Journal of Policy Modeling, Elsevier, vol. 28(2), pages 223-234, February.
  5. Dean Croushore & Tom Stark, 1999. "A real-time data set for marcoeconomists: does the data vintage matter?," Working Papers 99-21, Federal Reserve Bank of Philadelphia.
  6. Andrew Ang & Geert Bekaert & Min Wei, 2006. "Do macro variables, asset markets, or surveys forecast inflation better?," Finance and Economics Discussion Series 2006-15, Board of Governors of the Federal Reserve System (U.S.).
  7. John W. Galbraith & Greg Tkacz, 2007. "Forecast Content And Content Horizons For Some Important Macroeconomic Time Series," Departmental Working Papers 2007-01, McGill University, Department of Economics.
  8. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-84, March.
  9. Terasvirta, T & Anderson, H M, 1992. "Characterizing Nonlinearities in Business Cycles Using Smooth Transition Autoregressive Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 7(S), pages S119-36, Suppl. De.
  10. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268.
  11. Diebold, Francis X & Rudebusch, Glenn D, 1989. "Scoring the Leading Indicators," The Journal of Business, University of Chicago Press, vol. 62(3), pages 369-91, July.
  12. Hardle, W. & Mammen, E., 1990. "Comparing nonparametric versus parametric regression fits," CORE Discussion Papers 1990065, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
  13. Rudebusch, Glenn D. & Williams, John C., 2009. "Forecasting Recessions: The Puzzle of the Enduring Power of the Yield Curve," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 492-503.
  14. Linton, O. & Whang, Yoon-Jae, 2007. "The quantilogram: With an application to evaluating directional predictability," Journal of Econometrics, Elsevier, vol. 141(1), pages 250-282, November.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. John Galbraith & Simon van Norden, 2009. "Calibration and Resolution Diagnostics for Bank of England Density Forecasts," CIRANO Working Papers 2009s-36, CIRANO.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:cir:cirwor:2008s-28. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Webmaster).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.