IDEAS home Printed from https://ideas.repec.org/p/esj/esridp/302.html
   My bibliography  Save this paper

Evaluating Density Forecasts with Applications to ESPF

Author

Listed:
  • BAN Kanemi
  • KAWAGOE Masaaki
  • MATSUOKA Hideaki

Abstract

This paper evaluates density forecasts using micro data from the ESP forecast (ESPF), a monthly survey of Japanese professional forecasters. The ESPF has collected individual density forecasts since June 2008. We employ two approaches, Probability Integral Transform (PIT) and Ranked Probability Score (RPS). First, we apply Berkowitz’s (2001) test to individual density forecasts produced every June. We fail to reject the independency in FY 2010 and 2011 real GDP growth rates. As for CPI inflation rates, we reject the independency in all the samples during FY 2008 to 2011, but fail to reject it if the sample is limited to a half with better forecast performance. The result may ensure individual densities coincide with unobserved true data generation process of the actual outcomes. Second, we calculate RPS, following Kenny, Kostka, and Masera (2012), and compare the Mean Probability Distribution (MPD), the average of individual densities, with three benchmarks -- Uniform, Normal and Naïve distributions -- and individual density forecasts. The MPD turns out to be a “good” density: it beats the benchmarks in most cases and ranks about fifth out of around 35 participants every year. Subjective judgments added to the MPD are likely to deteriorate the performance in the case of CPI inflation rate, but to improve in the case of real GDP growth rate.

Suggested Citation

  • BAN Kanemi & KAWAGOE Masaaki & MATSUOKA Hideaki, 2013. "Evaluating Density Forecasts with Applications to ESPF," ESRI Discussion paper series 302, Economic and Social Research Institute (ESRI).
  • Handle: RePEc:esj:esridp:302
    as

    Download full text from publisher

    File URL: http://www.esri.go.jp/jp/archive/e_dis/e_dis302/e_dis302.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    2. Boero, Gianna & Smith, Jeremy & Wallis, Kenneth F., 2011. "Scoring rules and survey density forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 379-393.
    3. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    4. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    Full references (including those not matched with items on IDEAS)

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:esj:esridp:302. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (KAWAMOTO Takuma). General contact details of provider: http://edirc.repec.org/data/esrgvjp.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.