IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2608.07479.html

Marginally Useful: An Information-Gap Identity in Conformal Prediction

Author

Listed:
  • Peter Cotton

Abstract

Conformal prediction has been touted as a more formal, rigorous approach to adding uncertainty to a forecast. The sole objective of this note is to point out that rigor cuts both ways in the case of residual pooling, the technique used in the vast majority of conformal prediction applications. The fact that unconditional guarantee of coverage is provided is not in question, but we make clear, we believe for the first time, that there is an opposing guarantee too: a permanent gambit of logarithmic-score regret which no amount of data or tuning can subsequently reduce. We give the exact size of the sacrifice, and a financial reading of it as the growth rate of an oracle adversary betting against odds set by someone using residual pooling.

Suggested Citation

  • Peter Cotton, 2026. "Marginally Useful: An Information-Gap Identity in Conformal Prediction," Papers 2608.07479, arXiv.org, revised Aug 2026.
    • Handle: RePEc:arx:papers:2608.07479
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2608.07479
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jing Lei & Max G’Sell & Alessandro Rinaldo & Ryan J. Tibshirani & Larry Wasserman, 2018. "Distribution-Free Predictive Inference for Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1094-1111, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    2. Zhongjun Qu & Wendun Wang & Xiaomeng Zhang, 2025. "Prediction Intervals for Model Averaging," Papers 2510.16224, arXiv.org.
    3. Lazic, Ivan & Barà, Chiara & Iovino, Marta & Stramaglia, Sebastiano & Kasas-Lazetic, Karolina & Jakovljević, Nikša & Faes, Luca, 2026. "Information-theoretic quantification of high-order feature effects in classification problems," Chaos, Solitons & Fractals, Elsevier, vol. 204(C).
    4. Brian D. Williamson & Peter B. Gilbert & Marco Carone & Noah Simon, 2021. "Nonparametric variable importance assessment using machine learning techniques," Biometrics, The International Biometric Society, vol. 77(1), pages 9-22, March.
    5. Tengyuan Liang, 2022. "Universal prediction band via semi‐definite programming," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(4), pages 1558-1580, September.
    6. Victor Chernozhukov & Kaspar Wuthrich & Yinchu Zhu, 2019. "Distributional conformal prediction," Papers 1909.07889, arXiv.org, revised Aug 2021.
    7. Leying Guan, 2023. "Localized conformal prediction: a generalized inference framework for conformal prediction," Biometrika, Biometrika Trust, vol. 110(1), pages 33-50.
    8. Pedro Delicado & Daniel Peña, 2023. "Understanding complex predictive models with ghost variables," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 32(1), pages 107-145, March.
    9. Yichao Liu & Peter Fransson & Julian Heidecke & Prasad Liyanage & Jonas Wallin & Joacim Rocklöv, 2025. "An explainable covariate compartmental model for predicting the spatio-temporal patterns of dengue in Sri Lanka," PLOS Computational Biology, Public Library of Science, vol. 21(9), pages 1-19, September.
    10. MohammadHossein Bateni & Zahra Hadizadeh & MohammadTaghi Hajiaghayi & Mahdi JafariRaviz & Shayan Taherijam, 2026. "Networked Information Aggregation for Binary Classification," Papers 2605.01082, arXiv.org.
    11. Yan Liu & Ye Luo & Zigan Wang & Xiaowei Zhang, 2026. "Uncertainty-Adjusted Sorting for Asset Pricing with Machine Learning," Papers 2601.00593, arXiv.org.
    12. Kristin Blesch & David S. Watson & Marvin N. Wright, 2024. "Conditional feature importance for mixed data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 108(2), pages 259-278, June.
    13. Ontivero-Ortega, M. & Fania, A. & Lacalamita, A. & Bellotti, R. & Monaco, A. & Stramaglia, S., 2025. "Cooperative effects in feature importance of individual patterns: Application to air pollutants and Alzheimer’s disease," Chaos, Solitons & Fractals, Elsevier, vol. 201(P3).
    14. Lasanthi C. R. Pelawa Watagoda & David J. Olive, 2021. "Comparing six shrinkage estimators with large sample theory and asymptotically optimal prediction intervals," Statistical Papers, Springer, vol. 62(5), pages 2407-2431, October.
    15. Hu, Jianming & Luo, Qingxi & Tang, Jingwei & Heng, Jiani & Deng, Yuwen, 2022. "Conformalized temporal convolutional quantile regression networks for wind power interval forecasting," Energy, Elsevier, vol. 248(C).
    16. Solari, Aldo & Djordjilović, Vera, 2022. "Multi split conformal prediction," Statistics & Probability Letters, Elsevier, vol. 184(C).
    17. Behnam Mohammadi & Nikhil Malik & Tim Derdenger & Kannan Srinivasan, 2025. "Regulating Explainable Artificial Intelligence (XAI) May Harm Consumers," Marketing Science, INFORMS, vol. 44(3), pages 711-724, May.
    18. Diquigiovanni, Jacopo & Fontana, Matteo & Vantini, Simone, 2022. "Conformal prediction bands for multivariate functional data," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    19. Yinuo Zhang & Sophia Tang & Tong Chen & Elizabeth Mahood & Sophia Vincoff & Pranam Chatterjee, 2026. "PeptiVerse: A unified platform for therapeutic peptide property prediction," Nature Communications, Nature, vol. 17(1), pages 1-12, December.
    20. Anja Zgodic & Ray Bai & Jiajia Zhang & Yuan Wang & Christopher Rorden & Alexander C. McLain, 2026. "Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression," Computational Statistics, Springer, vol. 41(1), pages 1-23, January.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2608.07479. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.