IDEAS home Printed from https://ideas.repec.org/p/gwc/wpaper/2015-006.html
   My bibliography  Save this paper

A Nonparametric Approach to Identifying a Subset of Forecasters that Outperforms the Simple Average

Author

Listed:
  • Constantin Bürgi

    () (The George Washington University)

  • Tara M. Sinclair

    () (The George Washington University)

Abstract

Empirical studies in the forecast combination literature have shown that it is notoriously di!cult to improve upon the simple average despite the availability of optimal combination weights. In particular, historical performance-based combination approaches do not select forecasters that improve upon the simple average going forward. This paper shows that this is due to the high correlation among forecasters, which only by chance causes some individuals to have lower root mean squared errors (RMSE) than the simple average. We introduce a new nonparametric approach to eliminate forecasters who perform well based purely on chance as well as poor performers. This leaves a subset of forecasters with better performance in subsequent periods. It improves upon the simple average in the SPF for bond yields where some forecasters may be more likely to have specialized knowledge.

Suggested Citation

  • Constantin Bürgi & Tara M. Sinclair, 2015. "A Nonparametric Approach to Identifying a Subset of Forecasters that Outperforms the Simple Average," Working Papers 2015-006, The George Washington University, Department of Economics, Research Program on Forecasting.
  • Handle: RePEc:gwc:wpaper:2015-006
    as

    Download full text from publisher

    File URL: https://www2.gwu.edu/~forcpgm/2015-006.pdf
    File Function: First version, 2015
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    2. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    3. Yongchen Zhao, 2015. "Robustness of Forecast Combination in Unstable Environment: A Monte Carlo Study of Advanced Algorithms," Working Papers 2015-04, Towson University, Department of Economics, revised Dec 2015.
    4. Christian Kascha & Francesco Ravazzolo, 2010. "Combining inflation density forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(1-2), pages 231-250.
    5. Capistrán, Carlos & Timmermann, Allan, 2009. "Forecast Combination With Entry and Exit of Experts," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 428-440.
    6. Genre, Véronique & Kenny, Geoff & Meyler, Aidan & Timmermann, Allan, 2013. "Combining expert forecasts: Can anything beat the simple average?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 108-121.
    7. Lahiri Kajal & Yang Liu, 2016. "A non-linear forecast combination procedure for binary outcomes," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 20(4), pages 421-440, September.
    8. Todd E. Clark & Michael W. McCracken, 2010. "Averaging forecasts from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 5-29.
    9. Poncela, Pilar & Rodríguez, Julio & Sánchez-Mangas, Rocío & Senra, Eva, 2011. "Forecast combination through dimension reduction techniques," International Journal of Forecasting, Elsevier, vol. 27(2), pages 224-237, April.
    10. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    11. Conflitti, Cristina & De Mol, Christine & Giannone, Domenico, 2015. "Optimal combination of survey forecasts," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1096-1103.
    12. Lahiri, Kajal & Peng, Huaming & Zhao, Yongchen, 2015. "Testing the value of probability forecasts for calibrated combining," International Journal of Forecasting, Elsevier, vol. 31(1), pages 113-129.
    13. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    14. Batchelor, R A, 1990. "All Forecasters Are Equal," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 143-144, January.
    15. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    16. Kajal Lahiri & Huaming Peng & Yongchen Zhao, 2017. "Online learning and forecast combination in unbalanced panels," Econometric Reviews, Taylor & Francis Journals, vol. 36(1-3), pages 257-288, March.
    17. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    18. Gamber, Edward N. & Smith, Julie K. & McNamara, Dylan C., 2014. "Where is the Fed in the distribution of forecasters?," Journal of Policy Modeling, Elsevier, vol. 36(2), pages 296-312.
    19. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    20. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Forecast combination; Forecast evaluation; Multiple model comparisons; Real-time data; Survey of Professional Forecasters;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gwc:wpaper:2015-006. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Tara M. Sinclair). General contact details of provider: http://edirc.repec.org/data/degwuus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.