IDEAS home Printed from https://ideas.repec.org/p/zbw/sfb475/200026.html
   My bibliography  Save this paper

Hits-and-misses for the evaluation and combination of forecasts

Author

Listed:
  • Wenzel, Thomas

Abstract

Error measures for the evaluation of forecasts are usually based on the size of the forecast errors. Common measures are e.g. the Mean Squared Error (MSE), the Mean Absolute Deviation (MAD) or the Mean Absolute Percentage Error (MAPE). Alternative measures for the comparison of forecasts are turning points or hits-and-misses, where an indicator loss function is used to decide, if a forecast is of high quality or not. Here, we discuss the latter to obtain reliable combined forecasts.

Suggested Citation

  • Wenzel, Thomas, 2000. "Hits-and-misses for the evaluation and combination of forecasts," Technical Reports 2000,26, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    • Handle: RePEc:zbw:sfb475:200026
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/77272/2/2000-26.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ed Dammers, 1993. "Measurement in the ex post evaluation of forecasts," Quality & Quantity: International Journal of Methodology, Springer, vol. 27(1), pages 31-45, February.
    2. Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-533, October.
    3. Flores, Benito E., 1989. "The utilization of the Wilcoxon test to compare forecasting methods: A note," International Journal of Forecasting, Elsevier, vol. 5(4), pages 529-535.
    4. Flores, Benito E., 1986. "Use of the sign test to supplement the percentage better statistic," International Journal of Forecasting, Elsevier, vol. 2(4), pages 477-489.
    5. Wenzel, Thomas, 1998. "Pitman-closeness and the linear combination of multivariate forecasts," Technical Reports 1998,34, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    6. Wenzel, Thomas, 1999. "Combination of biased forecasts: Bias correction or bias based weights?," Technical Reports 1999,50, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    7. Cumby, Robert E. & Modest, David M., 1987. "Testing for market timing ability : A framework for forecast evaluation," Journal of Financial Economics, Elsevier, vol. 19(1), pages 169-189, September.
    8. Schneider, Carsten & Klapper, Matthias & Wenzel, Thomas, 1999. "An evaluation of forecasting methods and forecast combination methods in goods management systems," Technical Reports 1999,31, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    9. Thomas D. Russell & Everett E. Adam, Jr., 1987. "An Empirical Evaluation of Alternative Forecasting Combinations," Management Science, INFORMS, vol. 33(10), pages 1267-1276, October.
    10. Klapper, Matthias, 1998. "Combining German macro economic forecasts using rank-based techniques," Technical Reports 1998,19, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    11. Cicarelli, James, 1982. "A new method of evaluating the accuracy of economic forecasts," Journal of Macroeconomics, Elsevier, vol. 4(4), pages 469-475.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas Wenzel, 2001. "Hits-and-misses for the evaluation and combination of forecasts," Journal of Applied Statistics, Taylor & Francis Journals, vol. 28(6), pages 759-773.
    2. Blaskowitz, Oliver & Herwartz, Helmut, 2014. "Testing the value of directional forecasts in the presence of serial correlation," International Journal of Forecasting, Elsevier, vol. 30(1), pages 30-42.
    3. Fritsche, Ulrich & Pierdzioch, Christian & Rülke, Jan-Christoph & Stadtmann, Georg, 2015. "Forecasting the Brazilian real and the Mexican peso: Asymmetric loss, forecast rationality, and forecaster herding," International Journal of Forecasting, Elsevier, vol. 31(1), pages 130-139.
    4. Pesaran, M. Hashem & Timmermann, Allan, 2009. "Testing Dependence Among Serially Correlated Multicategory Variables," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 325-337.
    5. Garcia, Philip & Irwin, Scott H. & Leuthold, Raymond M. & Yang, Li, 1997. "The value of public information in commodity futures markets," Journal of Economic Behavior & Organization, Elsevier, vol. 32(4), pages 559-570, April.
    6. Mark T. Leung & An-Sing Chen, 2005. "Performance evaluation of neural network architectures: the case of predicting foreign exchange correlations," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 24(6), pages 403-420.
    7. Lim, Terence & Lo, Andrew W. & Merton, Robert C. & Scholes, Myron S., 2006. "The Derivatives Sourcebook," Foundations and Trends(R) in Finance, now publishers, vol. 1(5–6), pages 365-572, April.
    8. Stanislav Anatolyev, 2006. "Testing for predictability (in Russian)," Quantile, Quantile, issue 1, pages 39-42, September.
    9. Barnhart, Scott W. & Giannetti, Antoine, 2009. "Negative earnings, positive earnings and stock return predictability: An empirical examination of market timing," Journal of Empirical Finance, Elsevier, vol. 16(1), pages 70-86, January.
    10. Lang, Korbinian & Auer, Benjamin R., 2020. "The economic and financial properties of crude oil: A review," The North American Journal of Economics and Finance, Elsevier, vol. 52(C).
    11. Peter Christoffersen & Francis X. Diebold, 2002. "Financial Asset Returns, Market Timing, and Volatility Dynamics," CIRANO Working Papers 2002s-02, CIRANO.
    12. Oliver Blaskowitz & Helmut Herwartz, 2008. "Testing directional forecast value in the presence of serial correlation," SFB 649 Discussion Papers SFB649DP2008-073, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    13. Peter F. Christoffersen & Francis X. Diebold, 2006. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," Management Science, INFORMS, vol. 52(8), pages 1273-1287, August.
    14. Nicolosi, Gina & Peng, Liang & Zhu, Ning, 2009. "Do individual investors learn from their trading experience?," Journal of Financial Markets, Elsevier, vol. 12(2), pages 317-336, May.
    15. Hsin-Min Lu & Chia-Shang J. Chu, 2006. "Random walk hypothesis in exchange rate reconsidered," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(4), pages 275-290.
    16. Rapach, David E. & Wohar, Mark E., 2006. "The out-of-sample forecasting performance of nonlinear models of real exchange rate behavior," International Journal of Forecasting, Elsevier, vol. 22(2), pages 341-361.
    17. Ash, J. C. K. & Smyth, D. J. & Heravi, S. M., 1998. "Are OECD forecasts rational and useful?: a directional analysis," International Journal of Forecasting, Elsevier, vol. 14(3), pages 381-391, September.
    18. Jaehun Chung & Yongmiao Hong, 2007. "Model-free evaluation of directional predictability in foreign exchange markets," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(5), pages 855-889.
    19. Sadorsky, Perry, 2006. "Modeling and forecasting petroleum futures volatility," Energy Economics, Elsevier, vol. 28(4), pages 467-488, July.
    20. Helmut Herwartz & Leonardo Morales-Arias, 2009. "In-sample and out-of-sample properties of international stock return dynamics conditional on equilibrium pricing factors," The European Journal of Finance, Taylor & Francis Journals, vol. 15(1), pages 1-28.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:sfb475:200026. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/isdorde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.