IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2603.07458.html

ForeComp: An R Package for Comparing Predictive Accuracy Using Fixed-Smoothing Asymptotics

Author

Listed:
  • Minchul Shin
  • Nathan Schor

Abstract

We introduce ForeComp, an R package for comparing predictive accuracy using Diebold-Mariano type tests of equal predictive ability with standard and fixed smoothing inference. The package provides a common interface for loss differential based testing and includes Plot Tradeoff, a visual diagnostic for bandwidth sensitivity and the size-power tradeoff. We illustrate the toolkit with Survey of Professional Forecasters applications and Monte Carlo evidence on finite-sample performance.

Suggested Citation

  • Minchul Shin & Nathan Schor, 2026. "ForeComp: An R Package for Comparing Predictive Accuracy Using Fixed-Smoothing Asymptotics," Papers 2603.07458, arXiv.org.
    • Handle: RePEc:arx:papers:2603.07458
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2603.07458
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Laura Coroneo & Fabrizio Iacone, 2020. "Comparing predictive accuracy in small samples using fixed‐smoothing asymptotics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 391-409, June.
    2. Yixiao Sun, 2013. "A heteroskedasticity and autocorrelation robust F test using an orthonormal series variance estimator," Econometrics Journal, Royal Economic Society, vol. 16(1), pages 1-26, February.
    3. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    4. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    5. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2005. "A New Asymptotic Theory For Heteroskedasticity-Autocorrelation Robust Tests," Econometric Theory, Cambridge University Press, vol. 21(6), pages 1130-1164, December.
    6. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    7. Tom Stark, 2010. "Realistic evaluation of real-time forecasts in the Survey of Professional Forecasters," Research Rap Special Report, Federal Reserve Bank of Philadelphia, issue May.
    8. Whitney K. Newey & Kenneth D. West, 1994. "Automatic Lag Selection in Covariance Matrix Estimation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 631-653.
    9. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    10. Ibragimov, Rustam & Müller, Ulrich K., 2010. "t-Statistic Based Correlation and Heterogeneity Robust Inference," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(4), pages 453-468.
    11. Eben Lazarus & Daniel J. Lewis & James H. Stock & Mark W. Watson, 2018. "HAR Inference: Recommendations for Practice Rejoinder," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 574-575, October.
    12. Michael W. McCracken, 2019. "Tests of Conditional Predictive Ability: Some Simulation Evidence," Working Papers 2019-11, Federal Reserve Bank of St. Louis.
    13. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    14. Eben Lazarus & Daniel J. Lewis & James H. Stock & Mark W. Watson, 2018. "HAR Inference: Recommendations for Practice," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 541-559, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Minchul Shin, 2026. "An Auditable AI Agent Loop for Empirical Economics: A Case Study in Forecast Combination," Papers 2603.17381, arXiv.org, revised Mar 2026.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laura Coroneo & Fabrizio Iacone, 2020. "Comparing predictive accuracy in small samples using fixed‐smoothing asymptotics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 391-409, June.
    2. Coroneo, Laura & Iacone, Fabrizio & Profumo, Fabio, 2024. "Survey density forecast comparison in small samples," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1486-1504.
    3. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    4. Coroneo, Laura & Iacone, Fabrizio, 2025. "Testing for equal predictive accuracy with strong dependence," International Journal of Forecasting, Elsevier, vol. 41(3), pages 1073-1092.
    5. Coroneo, Laura & Iacone, Fabrizio & Paccagnini, Alessia & Santos Monteiro, Paulo, 2023. "Testing the predictive accuracy of COVID-19 forecasts," International Journal of Forecasting, Elsevier, vol. 39(2), pages 606-622.
    6. Casini, Alessandro & Perron, Pierre, 2024. "Prewhitened long-run variance estimation robust to nonstationarity," Journal of Econometrics, Elsevier, vol. 242(1).
    7. Hirukawa, Masayuki, 2023. "Robust Covariance Matrix Estimation in Time Series: A Review," Econometrics and Statistics, Elsevier, vol. 27(C), pages 36-61.
    8. Kolokotrones, Thomas & Stock, James H. & Walker, Christopher D., 2024. "Is Newey–West optimal among first-order kernels?," Journal of Econometrics, Elsevier, vol. 240(2).
    9. Laura Coroneo & Fabrizio Iacone, 2015. "Comparing predictive accuracy in small samples," Discussion Papers 15/15, Department of Economics, University of York.
    10. Yu‐Sheng Lai, 2022. "Use of high‐frequency data to evaluate the performance of dynamic hedging strategies," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 42(1), pages 104-124, January.
    11. Matei Demetrescu & Christoph Hanck & Robinson Kruse‐Becher, 2022. "Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 1010-1030, August.
    12. Federico Belotti & Alessandro Casini & Leopoldo Catania & Stefano Grassi & Pierre Perron, 2023. "Simultaneous bandwidths determination for DK-HAC estimators and long-run variance estimation in nonparametric settings," Econometric Reviews, Taylor & Francis Journals, vol. 42(3), pages 281-306, February.
    13. Eben Lazarus & Daniel J. Lewis & James H. Stock, 2021. "The Size‐Power Tradeoff in HAR Inference," Econometrica, Econometric Society, vol. 89(5), pages 2497-2516, September.
    14. Rho, Seunghwa & Vogelsang, Timothy J., 2021. "Inference in time series models using smoothed-clustered standard errors," Journal of Econometrics, Elsevier, vol. 224(1), pages 113-133.
    15. Jungbin Hwang & Gonzalo Valdés, 2025. "HAR Inference for Quantile Regression in Time Series," Working papers 2025-03, University of Connecticut, Department of Economics, revised Mar 2026.
    16. Pellatt, Daniel F. & Sun, Yixiao, 2023. "Asymptotic F test in regressions with observations collected at high frequency over long span," Journal of Econometrics, Elsevier, vol. 235(2), pages 1281-1309.
    17. Filip Stanek, 2021. "Optimal Out-of-Sample Forecast Evaluation under Stationarity," CERGE-EI Working Papers wp712, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    18. Christian Glocker & Serguei Kaniovski, 2022. "Macroeconometric forecasting using a cluster of dynamic factor models," Empirical Economics, Springer, vol. 63(1), pages 43-91, July.
    19. Pellatt , Daniel & Sun, Yixiao, 2020. "Asymptotic F test in Regressions with Observations Collected at High Frequency over Long Span," University of California at San Diego, Economics Working Paper Series qt19f0d9wz, Department of Economics, UC San Diego.
    20. Coroneo, Laura, 2026. "Forecasting for monetary policy," International Journal of Forecasting, Elsevier, vol. 42(1), pages 22-33.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2603.07458. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.