IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2608.23064.html

Sequentially valid inference for probabilistic inflation forecasts

Author

Listed:
  • Amadeo Grob
  • Maurizio Daniele
  • Johanna Ziegel

Abstract

Traditional statistical tests are poorly suited for the sequential evaluation of probabilistic forecast calibration. We address this limitation in macroeconomic forecasting by applying a new sequential testing method based on e-values. The e-value-based methodology enables anytime-valid inference. It allows practitioners to test against calibration continuously without invalidating statistical guarantees. To illustrate the framework's practical value, we apply it to probabilistic inflation forecasts for the United States, the Euro Area, and Switzerland. Our analysis shows that the sequential approach gives detailed insights into the timing and nature of forecast misspecification. We find these diagnostics are particularly insightful during major structural breaks. During these events, we find evidence against calibration that static, full-sample tests often miss. Therefore, this work shows that e-value-based tests are a practical method for the evaluation of forecast calibration in empirical macroeconomics.

Suggested Citation

  • Amadeo Grob & Maurizio Daniele & Johanna Ziegel, 2026. "Sequentially valid inference for probabilistic inflation forecasts," Papers 2608.23064, arXiv.org.
  • Handle: RePEc:arx:papers:2608.23064
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2608.23064
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    2. Glenn Shafer, 2021. "Testing by betting: A strategy for statistical and scientific communication," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(2), pages 407-431, April.
    3. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    4. Goulet Coulombe, Philippe & Leroux, Maxime & Stevanovic, Dalibor & Surprenant, Stéphane, 2021. "Macroeconomic data transformations matter," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1338-1354.
    5. Alexander Henzi & Johanna F Ziegel, 2022. "Valid sequential inference on probability forecast performance [A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems]," Biometrika, Biometrika Trust, vol. 109(3), pages 647-663.
    6. Litterman, Robert B, 1986. "Forecasting with Bayesian Vector Autoregressions-Five Years of Experience," Journal of Business & Economic Statistics, American Statistical Association, vol. 4(1), pages 25-38, January.
    7. Alexander Henzi & Johanna F Ziegel, 2022. "Correction to: ‘Valid sequential inference on probability forecast performance’ [Valid sequential inference on probability forecast performance]," Biometrika, Biometrika Trust, vol. 109(4), pages 1181-1182.
    8. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    9. Bai, Jushan & Ng, Serena, 2007. "Determining the Number of Primitive Shocks in Factor Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 52-60, January.
    10. Chahad, Mohammed & Martínez Hernández, Catalina & Page, Adrian & Hofmann-Drahonsky, Anna-Camilla, 2024. "An update on the accuracy of recent Eurosystem/ECB staff projections for short-term inflation," Economic Bulletin Boxes, European Central Bank, vol. 2.
    11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    12. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    13. Marta Bańbura & Michele Modugno, 2014. "Maximum Likelihood Estimation Of Factor Models On Datasets With Arbitrary Pattern Of Missing Data," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(1), pages 133-160, January.
    14. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    15. Koop, Gary & Korobilis, Dimitris, 2010. "Bayesian Multivariate Time Series Methods for Empirical Macroeconomics," Foundations and Trends(R) in Econometrics, now publishers, vol. 3(4), pages 267-358, July.
    16. Michael P. Clements, 2004. "Evaluating the Bank of England Density Forecasts of Inflation," Economic Journal, Royal Economic Society, vol. 114(498), pages 844-866, October.
    17. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
    18. Alexei Onatski, 2010. "Determining the Number of Factors from Empirical Distribution of Eigenvalues," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1004-1016, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    2. Bae, Juhee, 2024. "Factor-augmented forecasting in big data," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1660-1688.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Paponpat Taveeapiradeecharoen & Nattapol Aunsri, 2025. "Forecasting in small open emerging economies Evidence from Thailand," Papers 2509.14805, arXiv.org.
    5. Dimitris Korobilis, 2013. "Var Forecasting Using Bayesian Variable Selection," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 204-230, March.
    6. Gupta, Rangan & Kabundi, Alain, 2011. "A large factor model for forecasting macroeconomic variables in South Africa," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1076-1088, October.
    7. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    8. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2015. "Realtime nowcasting with a Bayesian mixed frequency model with stochastic volatility," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 837-862, October.
    9. Timo Dimitriadis & Marius Puke, 2026. "Statistical Inference for Score Decompositions," Papers 2603.04275, arXiv.org, revised Sep 2026.
    10. Yo Joong Choe & Aaditya Ramdas, 2024. "Comparing Sequential Forecasters," Operations Research, INFORMS, vol. 72(4), pages 1368-1387, July.
    11. Tom Boot & Bart Keijsers, 2025. "Diffusion index forecasts under weaker loadings: PCA, ridge regression, and random projections," Papers 2506.09575, arXiv.org.
    12. Martinez-Martin Jaime & Morris Richard & Onorante Luca & Piersanti Fabio Massimo, 2024. "Merging Structural and Reduced-Form Models for Forecasting," The B.E. Journal of Macroeconomics, De Gruyter, vol. 24(1), pages 399-437, January.
    13. Magnus Reif, 2020. "Macroeconomics, Nonlinearities, and the Business Cycle," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 87, September.
    14. Raviv, Eran & Bouwman, Kees E. & van Dijk, Dick, 2015. "Forecasting day-ahead electricity prices: Utilizing hourly prices," Energy Economics, Elsevier, vol. 50(C), pages 227-239.
    15. Dedu, Vasile & Stoica, Tiberiu, 2014. "The Impact of Monetaru Policy on the Romanian Economy," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(2), pages 71-86, June.
    16. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2010. "Forecasting with DSGE models," Working Paper Series 1185, European Central Bank.
    17. Karlsson, Sune & Mazur, Stepan & Nguyen, Hoang, 2023. "Vector autoregression models with skewness and heavy tails," Journal of Economic Dynamics and Control, Elsevier, vol. 146(C).
    18. repec:rim:rimwps:20-27 is not listed on IDEAS
    19. Sandra Eickmeier & Christina Ziegler, 2008. "How successful are dynamic factor models at forecasting output and inflation? A meta-analytic approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(3), pages 237-265.
    20. Dimitrios P. Louzis, 2017. "Macroeconomic and credit forecasts during the Greek crisis using Bayesian VARs," Empirical Economics, Springer, vol. 53(2), pages 569-598, September.
    21. Boni, Sara & Iseringhausen, Martin & Petrella, Ivan & Theodoridis, Konstantinos, 2026. "A survey-based measure of asymmetric macroeconomic risk in the euro area," European Economic Review, Elsevier, vol. 188(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2608.23064. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.