IDEAS home Printed from https://ideas.repec.org/p/ags/nc8191/285735.html

Forecast Evaluation: A Likelihood Scoring Method

Author

Listed:
  • Diersen, Matthew A.
  • Manfredo, Mark R.

Abstract

While many forecast evaluation techniques are available, most are designed for the end user of forecasts. Most statistical evaluation procedures rely on a particular loss function. Forecast evaluation procedures, such as mean squared error and mean absolute error, that have different underlying loss functions, may provide conflicting results. This paper develops a new approach of evaluating forecasts, a likelihood scoring method, that does not rely on a particular loss function. The method takes a Bayesian approach to forecast evaluation and uses information from forecast prediction intervals. This method is used to evaluate structural econometric and ARIMA forecasting models of quarterly hog price.

Suggested Citation

  • Diersen, Matthew A. & Manfredo, Mark R., 1998. "Forecast Evaluation: A Likelihood Scoring Method," 1981-1999 Conference Archive 285735, NCR-134/ NCCC-134 Applied Commodity Price Analysis, Forecasting, and Market Risk Management.
    • Handle: RePEc:ags:nc8191:285735
    DOI: 10.22004/ag.econ.285735
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/285735/files/confp22-98.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.285735?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, "undated". "Evaluating Density Forecasts," CARESS Working Papres 97-18, University of Pennsylvania Center for Analytic Research and Economics in the Social Sciences.
    2. Makridakis, Spyros & Hibon, Michele & Lusk, Ed & Belhadjali, Moncef, 1987. "Confidence intervals: An empirical investigation of the series in the M-competition," International Journal of Forecasting, Elsevier, vol. 3(3-4), pages 489-508.
    3. Jon A. Brandt, 1985. "Forecasting and Hedging: An Illustration of Risk Reduction in the Hog Industry," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 67(1), pages 24-31.
    4. Chatfield, Chris, 1993. "Calculating Interval Forecasts: Reply," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(2), pages 143-144, April.
    5. Chatfield, Chris, 1993. "Calculating Interval Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(2), pages 121-135, April.
    6. David A. Bessler & John L. Kling, 1989. "The Forecast and Policy Analysis," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 71(2), pages 503-506.
    7. Allen, P. Geoffrey, 1994. "Economic forecasting in agriculture," International Journal of Forecasting, Elsevier, vol. 10(1), pages 81-135, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kurtuluş, Ercan & Çetin, İsmail Bilge, 2020. "Analysis of modal shift potential towards intermodal transportation in short-distance inland container transport," Transport Policy, Elsevier, vol. 89(C), pages 24-37.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:lan:wpaper:470 is not listed on IDEAS
    2. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    3. Silvano Bordignon & Francesco Lisi, 2001. "Interval prediction for chaotic time series," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(3-4), pages 117-140.
    4. Francis X. Diebold & Jinyong Hahn & Anthony S. Tay, 1998. "Real-Time Multivariate Density Forecast Evaluation and Calibration: Monitoring the Risk of High-Frequency Returns on Foreign Exchange," New York University, Leonard N. Stern School Finance Department Working Paper Seires 98-079, New York University, Leonard N. Stern School of Business-.
    5. repec:lan:wpaper:425 is not listed on IDEAS
    6. repec:lan:wpaper:539557 is not listed on IDEAS
    7. Guy Melard & Jean-Michel Pasteels, 2000. "Automatic ARIMA modeling including interventions, using time series expert software," ULB Institutional Repository 2013/13744, ULB -- Universite Libre de Bruxelles.
    8. repec:lan:wpaper:413 is not listed on IDEAS
    9. Griffiths, William E. & Newton, Lisa S. & O'Donnell, Christopher J., 2010. "Predictive densities for models with stochastic regressors and inequality constraints: Forecasting local-area wheat yield," International Journal of Forecasting, Elsevier, vol. 26(2), pages 397-412, April.
    10. Berrin Aytac & S. Wu, 2013. "Characterization of demand for short life-cycle technology products," Annals of Operations Research, Springer, vol. 203(1), pages 255-277, March.
    11. Sulandari, Winita & Subanar, & Lee, Muhammad Hisyam & Rodrigues, Paulo Canas, 2020. "Indonesian electricity load forecasting using singular spectrum analysis, fuzzy systems and neural networks," Energy, Elsevier, vol. 190(C).
    12. Christis Katsouris, 2023. "Structural Analysis of Vector Autoregressive Models," Papers 2312.06402, arXiv.org, revised Feb 2024.
    13. Jing Li, 2021. "Block bootstrap prediction intervals for parsimonious first‐order vector autoregression," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(3), pages 512-527, April.
    14. Lee, Yun Shin & Scholtes, Stefan, 2014. "Empirical prediction intervals revisited," International Journal of Forecasting, Elsevier, vol. 30(2), pages 217-234.
    15. Ling He & Chenyi Hu, 2010. "Midpoint method and accuracy of variability forecasting," Empirical Economics, Springer, vol. 38(3), pages 705-715, June.
    16. Hansen, Bruce E., 2006. "Interval forecasts and parameter uncertainty," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 377-398.
    17. Wolfgang Nierhaus, 2013. "Konjunkturprognosen heute – Möglichkeiten und Probleme," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 66(01), pages 25-32, January.
    18. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, "undated". "Evaluating Density Forecasts," CARESS Working Papres 97-18, University of Pennsylvania Center for Analytic Research and Economics in the Social Sciences.
    19. Charles, Amelie & Darne, Olivier & Kim, Jae, 2016. "Stock Return Predictability: Evaluation based on Prediction Intervals," MPRA Paper 70143, University Library of Munich, Germany.
    20. Sizhe Chen & Han Lin Shang & Yang Yang, 2025. "Is the age pension in Australia sustainable and fair? Evidence from forecasting the old-age dependency ratio using the Hamilton-Perry model," Journal of Population Research, Springer, vol. 42(1), pages 1-27, March.
    21. Dhaoui, Iyad, 2015. "Climat des Affaires et Compétitivité de l’Entreprise Tunisienne Après la Révolution : Analyses et Perspectives [Business Climate and Competitiveness of the Tunisian Enterprise After the Revolution: Analysis and Perspectives]," MPRA Paper 87331, University Library of Munich, Germany.
    22. Li, Anzhi & Dorfman, Jeffrey H., 2014. "Composite Qualitative Forecasting of Futures Prices: Using One Commodity to Help Forecast Another," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 169790, Agricultural and Applied Economics Association.
    23. Li, Yushu & Andersson, Jonas, 2014. "A Likelihood Ratio and Markov Chain Based Method to Evaluate Density Forecasting," Discussion Papers 2014/12, Norwegian School of Economics, Department of Business and Management Science.
    24. Wolfgang Nierhaus, 2018. "Wirtschaftskonjunktur 2017: Prognose und Wirklichkeit," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 71(03), pages 35-42, February.

    More about this item

    Keywords

    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:nc8191:285735. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: http://www.farmdoc.illinois.edu/nccc134/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.