IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20131540.html
   My bibliography  Save this paper

Can macroeconomists forecast risk? Event-based evidence from the euro area SPF

Author

Listed:
  • Kenny, Geoff
  • Kostka, Thomas
  • Masera, Federico

Abstract

We propose methods to evaluate the risk assessments collected as part of the ECB Survey of Professional Forecasters (SPF). Our approach focuses on direction-of-change predictions as well as the prediction of relatively more extreme macroeconomic outcomes located in the upper and lower regions of the predictive densities. For inflation and GDP growth, we find such surveyed densities are informative about future direction of change. Regarding more extreme high and low outcome events, the surveys are really only informative about GDP growth outcomes and at short-horizons. The upper and lower regions of the predictive densities for inflation are much less informative. JEL Classification: C22, C53

Suggested Citation

  • Kenny, Geoff & Kostka, Thomas & Masera, Federico, 2013. "Can macroeconomists forecast risk? Event-based evidence from the euro area SPF," Working Paper Series 1540, European Central Bank.
  • Handle: RePEc:ecb:ecbwps:20131540
    Note: 339061
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecbwp1540.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    2. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    3. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    4. Juan Angel Garcia, 2003. "An introduction to the ECB’s survey of professional forecasters," Occasional Paper Series 08, European Central Bank.
    5. Andrade, P. & Ghysels, E. & Idier, J., 2012. "Tails of Inflation Forecasts and Tales of Monetary Policy," Working papers 407, Banque de France.
    6. Fred Joutz & Michael P. Clements & Herman O. Stekler, 2007. "An evaluation of the forecasts of the federal reserve: a pooled approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(1), pages 121-136.
    7. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    8. Don Harding & Adrian Pagan, 2010. "Can We Predict Recessions?," NCER Working Paper Series 69, National Centre for Econometric Research.
    9. Geoff Kenny & Thomas Kostka & Federico Masera, 2014. "How Informative are the Subjective Density Forecasts of Macroeconomists?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(3), pages 163-185, April.
    10. Murphy, Allan H. & Winkler, Robert L., 1992. "Diagnostic verification of probability forecasts," International Journal of Forecasting, Elsevier, vol. 7(4), pages 435-455, March.
    11. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    12. Carlos Bowles & Roberta Friz & Veronique Genre & Geoff Kenny & Aidan Meyler & Tuomas Rautanen, 2010. "An Evaluation of the Growth and Unemployment Forecasts in the ECB Survey of Professional Forecasters," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2010(2), pages 1-28.
    13. James Mitchell & Kenneth F. Wallis, 2011. "Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(6), pages 1023-1040, September.
    14. Genre, Véronique & Kenny, Geoff & Meyler, Aidan & Timmermann, Allan, 2013. "Combining expert forecasts: Can anything beat the simple average?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 108-121.
    15. Michael Clements, 2006. "Evaluating the survey of professional forecasters probability distributions of expected inflation based on derived event probability forecasts," Empirical Economics, Springer, vol. 31(1), pages 49-64, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    2. Ganics, Gergely & Odendahl, Florens, 2021. "Bayesian VAR forecasts, survey information, and structural change in the euro area," International Journal of Forecasting, Elsevier, vol. 37(2), pages 971-999.
    3. Paola Cerchiello & Paolo Giudici, 2014. "Conditional graphical models for systemic risk measurement," DEM Working Papers Series 087, University of Pavia, Department of Economics and Management.
    4. Raffaella Calabrese & Johan A. Elkink & Paolo S. Giudici, 2017. "Measuring bank contagion in Europe using binary spatial regression models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(12), pages 1503-1511, December.
    5. Galvao, Ana Beatriz & Garratt, Anthony & Mitchell, James, 2020. "Does Judgment Improve Macroeconomic Density Forecasts?," EMF Research Papers 33, Economic Modelling and Forecasting Group.
    6. Bańbura, Marta & Brenna, Federica & Paredes, Joan & Ravazzolo, Francesco, 2021. "Combining Bayesian VARs with survey density forecasts: does it pay off?," Working Paper Series 2543, European Central Bank.
    7. Galvão, Ana Beatriz & Garratt, Anthony & Mitchell, James, 2021. "Does judgment improve macroeconomic density forecasts?," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1247-1260.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    2. Bańbura, Marta & Brenna, Federica & Paredes, Joan & Ravazzolo, Francesco, 2021. "Combining Bayesian VARs with survey density forecasts: does it pay off?," Working Paper Series 2543, European Central Bank.
    3. Clements, Michael P, 2012. "Subjective and Ex Post Forecast Uncertainty : US Inflation and Output Growth," The Warwick Economics Research Paper Series (TWERPS) 995, University of Warwick, Department of Economics.
    4. Constantin Bürgi & Tara M. Sinclair, 2017. "A nonparametric approach to identifying a subset of forecasters that outperforms the simple average," Empirical Economics, Springer, vol. 53(1), pages 101-115, August.
    5. Conflitti, Cristina & De Mol, Christine & Giannone, Domenico, 2015. "Optimal combination of survey forecasts," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1096-1103.
    6. Joshua Abel & Robert Rich & Joseph Song & Joseph Tracy, 2016. "The Measurement and Behavior of Uncertainty: Evidence from the ECB Survey of Professional Forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 533-550, April.
    7. Fabian Krüger, 2017. "Survey-based forecast distributions for Euro Area growth and inflation: ensembles versus histograms," Empirical Economics, Springer, vol. 53(1), pages 235-246, August.
    8. Robert W. Rich & Joseph Tracy, 2017. "The behavior of uncertainty and disagreement and their roles in economic prediction: a panel analysis," Staff Reports 808, Federal Reserve Bank of New York.
    9. Sami Oinonen & Maritta Paloviita, 2017. "How Informative are Aggregated Inflation Expectations? Evidence from the ECB Survey of Professional Forecasters," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 13(2), pages 139-163, November.
    10. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    11. Baumann, Ursel & Darracq Pariès, Matthieu & Westermann, Thomas & Riggi, Marianna & Bobeica, Elena & Meyler, Aidan & Böninghausen, Benjamin & Fritzer, Friedrich & Trezzi, Riccardo & Jonckheere, Jana & , 2021. "Inflation expectations and their role in Eurosystem forecasting," Occasional Paper Series 264, European Central Bank.
    12. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    13. Meyler, Aidan, 2020. "Forecast performance in the ECB SPF: ability or chance?," Working Paper Series 2371, European Central Bank.
    14. Maritta Paloviita and Matti Viren, 2012. "Are individual survey expectations internally consistent?," Discussion Papers 77, Aboa Centre for Economics.
    15. Manzan, Sebastiano, 2021. "Are professional forecasters Bayesian?," Journal of Economic Dynamics and Control, Elsevier, vol. 123(C).
    16. Clements, Michael P., 2006. "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    17. Magdalena Grothe & Aidan Meyler, 2018. "Inflation Forecasts: Are Market-Based and Survey-Based Measures Informative?," International Journal of Financial Research, International Journal of Financial Research, Sciedu Press, vol. 9(1), pages 171-188, January.
    18. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    19. Ambrocio, Gene, 2017. "The real effects of overconfidence and fundamental uncertainty shocks," Research Discussion Papers 37/2017, Bank of Finland.
    20. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.

    More about this item

    Keywords

    calibration error; forecast evaluation; probability forecasts; Survey of Professional Forecasters;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20131540. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/emieude.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.