IDEAS home Printed from https://ideas.repec.org/a/jec/journl/v5y2009i2p219-240.html
   My bibliography  Save this article

Assessing the Accuracy of Event Forecasts

Author

Listed:
  • Ching-Chuan Tsong

    (Department of Economics, National Chi Nan University, Taiwan)

Abstract

Event forecasts, often generated from estimated econometric models, comprise a binary time series. In empirical finance, the market timing test proposed by Henricksson and Merton (1981) is probably the most popular method to assess the accuracy of these forecasts. Unfortunately, event forecasts and/or realizations are serially correlated, violating the independent identical distributed (IID) assumption. Consequently, the market timing test has an inflated size that can lead to doubtful empirical results. We find that the heteroskedasticity- autocorrelation (HAC) robust t-test with fixed-b asymptotics in Kiefer and Vogelsang (2005) and with the empirical distribution obtained using the naive block bootstrap can overcome this problem. As compared to several extant testing methods, simulation results reveal that the empirical size of these two testing procedures is quite close to the nominal size in finite samples. An empirical study is performed to demonstrate the usefulness of the naive block bootstrap.

Suggested Citation

  • Ching-Chuan Tsong, 2009. "Assessing the Accuracy of Event Forecasts," Journal of Economics and Management, College of Business, Feng Chia University, Taiwan, vol. 5(2), pages 219-240, July.
    • Handle: RePEc:jec:journl:v:5:y:2009:i:2:p:219-240
    as

    Download full text from publisher

    File URL: http://www.jem.org.tw/content/pdf/Vol.5No.2/04.pdf
    Download Restriction: no
    File URL: http://www.jem.org.tw/content/abstract/Vol.5No.2/English/04.htm
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Engel, Charles, 1994. "Can the Markov switching model forecast exchange rates?," Journal of International Economics, Elsevier, vol. 36(1-2), pages 151-165, February.
    2. Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-533, October.
    3. Joon Y. Park, 2003. "Bootstrap Unit Root Tests," Econometrica, Econometric Society, vol. 71(6), pages 1845-1895, November.
    4. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    5. Breen, William & Glosten, Lawrence R & Jagannathan, Ravi, 1989. " Economic Significance of Predictable Variations in Stock Index Returns," Journal of Finance, American Finance Association, vol. 44(5), pages 1177-1189, December.
    6. Dimitris Politis & Halbert White, 2004. "Automatic Block-Length Selection for the Dependent Bootstrap," Econometric Reviews, Taylor & Francis Journals, vol. 23(1), pages 53-70.
    7. Lahiri, Soumendra Nath, 1996. "On Edgeworth Expansion and Moving Block Bootstrap for StudentizedM-Estimators in Multiple Linear Regression Models," Journal of Multivariate Analysis, Elsevier, vol. 56(1), pages 42-59, January.
    8. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2005. "A New Asymptotic Theory For Heteroskedasticity-Autocorrelation Robust Tests," Econometric Theory, Cambridge University Press, vol. 21(6), pages 1130-1164, December.
    9. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2002. "Heteroskedasticity-Autocorrelation Robust Testing Using Bandwidth Equal To Sample Size," Econometric Theory, Cambridge University Press, vol. 18(6), pages 1350-1366, December.
    10. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peter Calkins & Songsak Sriboonchitta & Aree Wiboonpongse, 2009. "Econometric Advances in the Service of Macroeconomic Prediction and Planning: An Overview," Journal of Economics and Management, College of Business, Feng Chia University, Taiwan, vol. 5(2), pages 159-166, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yixiao Sun & Peter C.B. Phillips, 2008. "Optimal Bandwidth Choice for Interval Estimation in GMM Regression," Cowles Foundation Discussion Papers 1661, Cowles Foundation for Research in Economics, Yale University.
    2. Blaskowitz, Oliver & Herwartz, Helmut, 2014. "Testing the value of directional forecasts in the presence of serial correlation," International Journal of Forecasting, Elsevier, vol. 30(1), pages 30-42.
    3. Paulo M. D. C. Parente & Richard J. Smith, 2021. "Quasi‐maximum likelihood and the kernel block bootstrap for nonlinear dynamic models," Journal of Time Series Analysis, Wiley Blackwell, vol. 42(4), pages 377-405, July.
    4. Matei Demetrescu & Christoph Hanck & Robinson Kruse‐Becher, 2022. "Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 1010-1030, August.
    5. Sun, Yixiao X & Phillips, Peter C. B. & Jin, Sainan, 2005. "Optimal Bandwidth Selection in Heteroskedasticity-Autocorrelation Robust Testing∗," University of California at San Diego, Economics Working Paper Series qt16b3j2hd, Department of Economics, UC San Diego.
    6. Christian A. Vossler, 2013. "Analyzing repeated-game economics experiments: robust standard errors for panel data with serial correlation," Chapters, in: John A. List & Michael K. Price (ed.), Handbook on Experimental Economics and the Environment, chapter 3, pages 89-112, Edward Elgar Publishing.
    7. Pötscher, Benedikt M. & Preinerstorfer, David, 2017. "Further Results on Size and Power of Heteroskedasticity and Autocorrelation Robust Tests, with an Application to Trend Testing," MPRA Paper 81053, University Library of Munich, Germany.
    8. Kim, Min Seong & Sun, Yixiao & Yang, Jingjing, 2017. "A fixed-bandwidth view of the pre-asymptotic inference for kernel smoothing with time series data," Journal of Econometrics, Elsevier, vol. 197(2), pages 298-322.
    9. Sun, Yixiao, 2014. "Let’s fix it: Fixed-b asymptotics versus small-b asymptotics in heteroskedasticity and autocorrelation robust inference," Journal of Econometrics, Elsevier, vol. 178(P3), pages 659-677.
    10. Martínez-Iriarte, Julián & Sun, Yixiao & Wang, Xuexin, 2020. "Asymptotic F tests under possibly weak identification," Journal of Econometrics, Elsevier, vol. 218(1), pages 140-177.
    11. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    12. Eben Lazarus & Daniel J. Lewis & James H. Stock, 2021. "The Size‐Power Tradeoff in HAR Inference," Econometrica, Econometric Society, vol. 89(5), pages 2497-2516, September.
    13. Kim, Min Seong & Sun, Yixiao, 2013. "Heteroskedasticity and spatiotemporal dependence robust inference for linear panel models with fixed effects," Journal of Econometrics, Elsevier, vol. 177(1), pages 85-108.
    14. Surajit Ray & N. E. Savin, 2008. "The performance of heteroskedasticity and autocorrelation robust tests: a Monte Carlo study with an application to the three-factor Fama-French asset-pricing model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(1), pages 91-109.
    15. Yixiao Sun & Xuexin Wang, 2019. "An Asymptotically F-Distributed Chow Test in the Presence of Heteroscedasticity and Autocorrelation," Papers 1911.03771, arXiv.org.
    16. Sun, Yixiao & Yang, Jingjing, 2020. "Testing-optimal kernel choice in HAR inference," Journal of Econometrics, Elsevier, vol. 219(1), pages 123-136.
    17. Hwang, Jungbin & Sun, Yixiao, 2018. "Should we go one step further? An accurate comparison of one-step and two-step procedures in a generalized method of moments framework," Journal of Econometrics, Elsevier, vol. 207(2), pages 381-405.
    18. Cho, Cheol-Keun & Amsler, Christine & Schmidt, Peter, 2015. "A test of the null of integer integration against the alternative of fractional integration," Journal of Econometrics, Elsevier, vol. 187(1), pages 217-237.
    19. Politis, D N, 2009. "Higher-Order Accurate, Positive Semi-definite Estimation of Large-Sample Covariance and Spectral Density Matrices," University of California at San Diego, Economics Working Paper Series qt66w826hz, Department of Economics, UC San Diego.
    20. Kim, Min Seong & Sun, Yixiao, 2011. "Spatial heteroskedasticity and autocorrelation consistent estimation of covariance matrix," Journal of Econometrics, Elsevier, vol. 160(2), pages 349-371, February.

    More about this item

    Keywords

    naive block bootstrap; HAC robust test; market timing test;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jec:journl:v:5:y:2009:i:2:p:219-240. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Yi-Ju Su (email available below). General contact details of provider: https://edirc.repec.org/data/cbfcutw.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.