IDEAS home Printed from https://ideas.repec.org/p/fip/fedpwp/103596.html

ForeComp: An R Package for Comparing Predictive Accuracy Using Fixed-Smoothing Asymptotics

Author

Listed:
  • Nathan Schor
  • Minchul Shin

Abstract

We introduce ForeComp, an R package for comparing predictive accuracy using Diebold–Mariano type tests of equal predictive ability with standard and fixed-smoothing inference. The package provides a common interface for loss-differential based testing and includes Plot Tradeoff, a visual diagnostic for bandwidth sensitivity and the size–power tradeoff. We illustrate the toolkit with Survey of Professional Forecasters applications and Monte Carlo evidence on finite-sample performance.

Suggested Citation

  • Nathan Schor & Minchul Shin, 2026. "ForeComp: An R Package for Comparing Predictive Accuracy Using Fixed-Smoothing Asymptotics," Working Papers 26-38, Federal Reserve Bank of Philadelphia.
    • Handle: RePEc:fip:fedpwp:103596
    DOI: 10.21799/frbp.wp.2026.38
    as

    Download full text from publisher

    File URL: https://www.philadelphiafed.org/-/media/frbp/assets/working-papers/2026/wp26-38.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.21799/frbp.wp.2026.38?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Laura Coroneo & Fabrizio Iacone, 2020. "Comparing predictive accuracy in small samples using fixed‐smoothing asymptotics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 391-409, June.
    2. Yixiao Sun, 2013. "A heteroskedasticity and autocorrelation robust F test using an orthonormal series variance estimator," Econometrics Journal, Royal Economic Society, vol. 16(1), pages 1-26, February.
    3. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    4. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    5. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2005. "A New Asymptotic Theory For Heteroskedasticity-Autocorrelation Robust Tests," Econometric Theory, Cambridge University Press, vol. 21(6), pages 1130-1164, December.
    6. Tom Stark, 2010. "Realistic evaluation of real-time forecasts in the Survey of Professional Forecasters," Research Rap Special Report, Federal Reserve Bank of Philadelphia, issue May.
    7. Whitney K. Newey & Kenneth D. West, 1994. "Automatic Lag Selection in Covariance Matrix Estimation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 631-653.
    8. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    9. Ibragimov, Rustam & Müller, Ulrich K., 2010. "t-Statistic Based Correlation and Heterogeneity Robust Inference," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(4), pages 453-468.
    10. Eben Lazarus & Daniel J. Lewis & James H. Stock & Mark W. Watson, 2018. "HAR Inference: Recommendations for Practice Rejoinder," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 574-575, October.
    11. Michael W. McCracken, 2019. "Tests of Conditional Predictive Ability: Some Simulation Evidence," Working Papers 2019-11, Federal Reserve Bank of St. Louis.
    12. Eben Lazarus & Daniel J. Lewis & James H. Stock & Mark W. Watson, 2018. "HAR Inference: Recommendations for Practice," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 541-559, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Minchul Shin & Nathan Schor, 2026. "ForeComp: An R Package for Comparing Predictive Accuracy Using Fixed-Smoothing Asymptotics," Papers 2603.07458, arXiv.org.
    2. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    3. Laura Coroneo & Fabrizio Iacone, 2020. "Comparing predictive accuracy in small samples using fixed‐smoothing asymptotics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 391-409, June.
    4. Coroneo, Laura & Iacone, Fabrizio & Profumo, Fabio, 2024. "Survey density forecast comparison in small samples," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1486-1504.
    5. Hirukawa, Masayuki, 2023. "Robust Covariance Matrix Estimation in Time Series: A Review," Econometrics and Statistics, Elsevier, vol. 27(C), pages 36-61.
    6. Coroneo, Laura & Iacone, Fabrizio, 2025. "Testing for equal predictive accuracy with strong dependence," International Journal of Forecasting, Elsevier, vol. 41(3), pages 1073-1092.
    7. Coroneo, Laura & Iacone, Fabrizio & Paccagnini, Alessia & Santos Monteiro, Paulo, 2023. "Testing the predictive accuracy of COVID-19 forecasts," International Journal of Forecasting, Elsevier, vol. 39(2), pages 606-622.
    8. Kolokotrones, Thomas & Stock, James H. & Walker, Christopher D., 2024. "Is Newey–West optimal among first-order kernels?," Journal of Econometrics, Elsevier, vol. 240(2).
    9. Laura Coroneo & Fabrizio Iacone, 2015. "Comparing predictive accuracy in small samples," Discussion Papers 15/15, Department of Economics, University of York.
    10. Federico Belotti & Alessandro Casini & Leopoldo Catania & Stefano Grassi & Pierre Perron, 2023. "Simultaneous bandwidths determination for DK-HAC estimators and long-run variance estimation in nonparametric settings," Econometric Reviews, Taylor & Francis Journals, vol. 42(3), pages 281-306, February.
    11. Eben Lazarus & Daniel J. Lewis & James H. Stock, 2021. "The Size‐Power Tradeoff in HAR Inference," Econometrica, Econometric Society, vol. 89(5), pages 2497-2516, September.
    12. Liyu Dou, 2024. "Optimal HAR inference," Quantitative Economics, Econometric Society, vol. 15(4), pages 1107-1149, November.
    13. Rho, Seunghwa & Vogelsang, Timothy J., 2021. "Inference in time series models using smoothed-clustered standard errors," Journal of Econometrics, Elsevier, vol. 224(1), pages 113-133.
    14. Jungbin Hwang & Gonzalo Valdés, 2025. "HAR Inference for Quantile Regression in Time Series," Working papers 2025-03, University of Connecticut, Department of Economics, revised Sep 2026.
    15. Pellatt, Daniel F. & Sun, Yixiao, 2023. "Asymptotic F test in regressions with observations collected at high frequency over long span," Journal of Econometrics, Elsevier, vol. 235(2), pages 1281-1309.
    16. Casini, Alessandro & Perron, Pierre, 2024. "Prewhitened long-run variance estimation robust to nonstationarity," Journal of Econometrics, Elsevier, vol. 242(1).
    17. Pellatt , Daniel & Sun, Yixiao, 2020. "Asymptotic F test in Regressions with Observations Collected at High Frequency over Long Span," University of California at San Diego, Economics Working Paper Series qt19f0d9wz, Department of Economics, UC San Diego.
    18. Casini, Alessandro, 2024. "The fixed-b limiting distribution and the ERP of HAR tests under nonstationarity," Journal of Econometrics, Elsevier, vol. 238(2).
    19. Jungbin Hwang & Gonzalo Valdés, 2020. "Low Frequency Cointegrating Regression in the Presence of Local to Unity Regressors and Unknown Form of Serial Dependence," Working papers 2020-03, University of Connecticut, Department of Economics, revised Aug 2020.
    20. Ulrich Hounyo & Min Seong Kim, 2025. "Robust Two-Sample Inference under Serial Dependence," Papers 2512.11259, arXiv.org, revised Jul 2026.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedpwp:103596. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Beth Paul (email available below). General contact details of provider: https://edirc.repec.org/data/frbphus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.