IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20141679.html
   My bibliography  Save this paper

Density characteristics and density forecast performance: a panel analysis

Author

Listed:
  • Kenny, Geoff
  • Kostka, Thomas
  • Masera, Federico

Abstract

In this paper, we exploit micro data from the ECB Survey of Professional Forecasters (SPF) to examine the link between the characteristics of macroeconomic density forecasts (such as their location, spread, skewness and tail risk) and density forecast performance. Controlling for the effects of common macroeconomic shocks, we apply cross-sectional and fixed effect panel regressions linking such density characteristics and density forecast performance. Our empirical results suggest that many macroeconomic experts could systematically improve their density performance by correcting a downward bias in their variances. Aside from this shortcoming in second moment characteristics of the individual densities, other higher moment features, such as skewness or variation in the degree of probability mass given to the tails of the predictive distributions tend - as a rule - not to contribute significantly to enhancing individual density forecast performance. JEL Classification: C22, C53

Suggested Citation

  • Kenny, Geoff & Kostka, Thomas & Masera, Federico, 2014. "Density characteristics and density forecast performance: a panel analysis," Working Paper Series 1679, European Central Bank.
    • Handle: RePEc:ecb:ecbwps:20141679
    Note: 339061
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecbwp1679.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    2. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Can Macroeconomists Forecast Risk? Event-Based Evidence from the Euro-Area SPF," International Journal of Central Banking, International Journal of Central Banking, vol. 11(4), pages 1-46, December.
    3. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    4. Lahiri, Kajal & Teigland, Christie & Zaporowski, Mark, 1988. "Interest Rates and the Subjective Probability Distribution of Inflation Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 20(2), pages 233-248, May.
    5. Griliches, Zvi & Hausman, Jerry A., 1986. "Errors in variables in panel data," Journal of Econometrics, Elsevier, vol. 31(1), pages 93-118, February.
    6. Garratt, Anthony & Mitchell, James & Vahey, Shaun P., 2014. "Probability Forecasting for Inflation Warnings from the Federal Reserve," EMF Research Papers 07, Economic Modelling and Forecasting Group.
    7. Geoff Kenny & Thomas Kostka & Federico Masera, 2014. "How Informative are the Subjective Density Forecasts of Macroeconomists?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(3), pages 163-185, April.
    8. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    9. Genre, Véronique & Kenny, Geoff & Meyler, Aidan & Timmermann, Allan, 2013. "Combining expert forecasts: Can anything beat the simple average?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 108-121.
    10. Casillas-Olvera, Gabriel & Bessler, David A., 2006. "Probability forecasting and central bank accountability," Journal of Policy Modeling, Elsevier, vol. 28(2), pages 223-234, February.
    11. Anthony Tay & Kenneth F. Wallis, 2000. "Density Forecasting: A Survey," Econometric Society World Congress 2000 Contributed Papers 0370, Econometric Society.
    12. Engle, Robert F. & White (the late), Halbert (ed.), 1999. "Cointegration, Causality, and Forecasting: Festschrift in Honour of Clive W. J. Granger," OUP Catalogue, Oxford University Press, number 9780198296836.
    13. Cristina Conflitti, 2012. "Measuring Uncertainty and Disagreement in the European Survey of Professional Forecasters," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2011(2), pages 69-103.
    14. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    15. Malte Knüppel & Guido Schultefrankenfeld, 2012. "How Informative Are Central Bank Assessments of Macroeconomic Risks?," International Journal of Central Banking, International Journal of Central Banking, vol. 8(3), pages 87-139, September.
    16. Michael Clements, 2006. "Evaluating the survey of professional forecasters probability distributions of expected inflation based on derived event probability forecasts," Empirical Economics, Springer, vol. 31(1), pages 49-64, March.
    17. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    18. Boero, Gianna & Smith, Jeremy & Wallis, Kenneth F., 2011. "Scoring rules and survey density forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 379-393.
    19. Andrade, P. & Ghysels, E. & Idier, J., 2012. "Tails of Inflation Forecasts and Tales of Monetary Policy," Working papers 407, Banque de France.
    20. Hsiao,Cheng & Pesaran,M. Hashem & Lahiri,Kajal & Lee,Lung Fei (ed.), 1999. "Analysis of Panels and Limited Dependent Variable Models," Cambridge Books, Cambridge University Press, number 9780521631693.
    21. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    22. Carlos Bowles & Roberta Friz & Veronique Genre & Geoff Kenny & Aidan Meyler & Tuomas Rautanen, 2010. "An Evaluation of the Growth and Unemployment Forecasts in the ECB Survey of Professional Forecasters," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2010(2), pages 1-28.
    23. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive Density Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 5, pages 197-284, Elsevier.
    24. Kajal Lahiri & J. George Wang, 2007. "The value of probability forecasts as predictors of cyclical downturns," Applied Economics Letters, Taylor & Francis Journals, vol. 14(1), pages 11-14.
    25. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    26. Lahiri, Kajal & Wang, J. George, 2013. "Evaluating probability forecasts for GDP declines using alternative methodologies," International Journal of Forecasting, Elsevier, vol. 29(1), pages 175-190.
    27. Robert Rich & Joseph Tracy, 2010. "The Relationships among Expected Inflation, Disagreement, and Uncertainty: Evidence from Matched Point and Density Forecasts," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 200-207, February.
    28. Giordani, Paolo & Soderlind, Paul, 2006. "Is there evidence of pessimism and doubt in subjective distributions? Implications for the equity premium puzzle," Journal of Economic Dynamics and Control, Elsevier, vol. 30(6), pages 1027-1043, June.
    29. James Mitchell & Kenneth F. Wallis, 2011. "Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(6), pages 1023-1040, September.
    30. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ambrocio, Gene, 2017. "The real effects of overconfidence and fundamental uncertainty shocks," Research Discussion Papers 37/2017, Bank of Finland.
    2. repec:zbw:bofrdp:2017_037 is not listed on IDEAS
    3. Constantin Bürgi & Tara M. Sinclair, 2017. "A nonparametric approach to identifying a subset of forecasters that outperforms the simple average," Empirical Economics, Springer, vol. 53(1), pages 101-115, August.
    4. Manzan, Sebastiano, 2021. "Are professional forecasters Bayesian?," Journal of Economic Dynamics and Control, Elsevier, vol. 123(C).
    5. Constantin Rudolf Salomo Bürgi, 2023. "How to deal with missing observations in surveys of professional forecasters," Journal of Applied Economics, Taylor & Francis Journals, vol. 26(1), pages 2185975-218, December.
    6. Victor Lopez-Perez, 2016. "Macroeconomic Forecast Uncertainty In The Euro Area," Equilibrium. Quarterly Journal of Economics and Economic Policy, Institute of Economic Research, vol. 11(1), pages 9-41, March.
    7. Jonas Dovern & Geoff Kenny, 2020. "Anchoring Inflation Expectations in Unconventional Times: Micro Evidence for the Euro Area," International Journal of Central Banking, International Journal of Central Banking, vol. 16(5), pages 309-347, October.
    8. Sami Oinonen & Maritta Paloviita, 2017. "How Informative are Aggregated Inflation Expectations? Evidence from the ECB Survey of Professional Forecasters," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 13(2), pages 139-163, November.
    9. López-Pérez, Víctor, 2016. "Does uncertainty affect non-response to the European Central Bank's survey of professional forecasters?," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 10, pages 1-47.
    10. repec:zbw:bofrdp:037 is not listed on IDEAS
    11. Alexander Glas & Matthias Hartmann, 2022. "Uncertainty measures from partially rounded probabilistic forecast surveys," Quantitative Economics, Econometric Society, vol. 13(3), pages 979-1022, July.
    12. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    13. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Can Macroeconomists Forecast Risk? Event-Based Evidence from the Euro-Area SPF," International Journal of Central Banking, International Journal of Central Banking, vol. 11(4), pages 1-46, December.
    14. Fabian Krüger, 2017. "Survey-based forecast distributions for Euro Area growth and inflation: ensembles versus histograms," Empirical Economics, Springer, vol. 53(1), pages 235-246, August.
    15. Glas, Alexander, 2020. "Five dimensions of the uncertainty–disagreement linkage," International Journal of Forecasting, Elsevier, vol. 36(2), pages 607-627.
    16. Kenny, Geoff & Dovern, Jonas, 2017. "The long-term distribution of expected inflation in the euro area: what has changed since the great recession?," Working Paper Series 1999, European Central Bank.
    17. Bańbura, Marta & Brenna, Federica & Paredes, Joan & Ravazzolo, Francesco, 2021. "Combining Bayesian VARs with survey density forecasts: does it pay off?," Working Paper Series 2543, European Central Bank.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Can Macroeconomists Forecast Risk? Event-Based Evidence from the Euro-Area SPF," International Journal of Central Banking, International Journal of Central Banking, vol. 11(4), pages 1-46, December.
    2. Clements, Michael P, 2012. "Subjective and Ex Post Forecast Uncertainty : US Inflation and Output Growth," The Warwick Economics Research Paper Series (TWERPS) 995, University of Warwick, Department of Economics.
    3. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    4. Joshua Abel & Robert Rich & Joseph Song & Joseph Tracy, 2016. "The Measurement and Behavior of Uncertainty: Evidence from the ECB Survey of Professional Forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 533-550, April.
    5. repec:zbw:bofrdp:037 is not listed on IDEAS
    6. Ambrocio, Gene, 2017. "The real effects of overconfidence and fundamental uncertainty shocks," Research Discussion Papers 37/2017, Bank of Finland.
    7. repec:zbw:bofrdp:2017_037 is not listed on IDEAS
    8. Alexander Glas & Matthias Hartmann, 2022. "Uncertainty measures from partially rounded probabilistic forecast surveys," Quantitative Economics, Econometric Society, vol. 13(3), pages 979-1022, July.
    9. Clements, Michael P., 2006. "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    10. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    11. Pilar Poncela & Eva Senra, 2017. "Measuring uncertainty and assessing its predictive power in the euro area," Empirical Economics, Springer, vol. 53(1), pages 165-182, August.
    12. Kajal Lahiri & Xuguang Sheng, 2010. "Measuring forecast uncertainty by disagreement: The missing link," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 514-538.
    13. Casey, Eddie, 2021. "Are professional forecasters overconfident?," International Journal of Forecasting, Elsevier, vol. 37(2), pages 716-732.
    14. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    15. Glas, Alexander, 2020. "Five dimensions of the uncertainty–disagreement linkage," International Journal of Forecasting, Elsevier, vol. 36(2), pages 607-627.
    16. Michael P. Clements, 2022. "Forecaster Efficiency, Accuracy, and Disagreement: Evidence Using Individual‐Level Survey Data," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 54(2-3), pages 537-568, March.
    17. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    18. Kajal Lahiri & Huaming Peng & Xuguang Simon Sheng, 2022. "Measuring Uncertainty of a Combined Forecast and Some Tests for Forecaster Heterogeneity," Advances in Econometrics, in: Essays in Honor of M. Hashem Pesaran: Prediction and Macro Modeling, volume 43, pages 29-50, Emerald Group Publishing Limited.
    19. Glas, Alexander & Hartmann, Matthias, 2016. "Inflation uncertainty, disagreement and monetary policy: Evidence from the ECB Survey of Professional Forecasters," Journal of Empirical Finance, Elsevier, vol. 39(PB), pages 215-228.
    20. Clements, Michael P., 2008. "Rounding of probability forecasts : The SPF forecast probabilities of negative output growth," The Warwick Economics Research Paper Series (TWERPS) 869, University of Warwick, Department of Economics.
    21. Manzan, Sebastiano, 2021. "Are professional forecasters Bayesian?," Journal of Economic Dynamics and Control, Elsevier, vol. 123(C).
    22. Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 1818, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 16 Nov 2020.

    More about this item

    Keywords

    density forecasting; forecast evaluation; Panel data; survey of professional forecasters;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20141679. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.