IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v66y2025i3d10.1007_s00362-025-01682-1.html
   My bibliography  Save this article

A small-sample Bayesian information criterion that does not overstate the evidence, with an application to calibrating p-values from likelihood-ratio tests

Author

Listed:
  • David R. Bickel

    (University of North Carolina at Greensboro)

Abstract

This paper proposes a simple correction to the Bayesian information criterion (BIC) for small samples to ensure that it neither overstates nor understates the evidence against a null hypothesis or other tested model. The new correction raises the likelihood ratio in the BIC to the power of 1 minus the reciprocal of the sample size ( $$1-1/\textrm{n}, \textrm{n}>1$$ 1 - 1 / n , n > 1 ). That is equivalent to multiplying the loglikelihood term of the BIC by a factor of $$1-1/\textrm{n}.$$ 1 - 1 / n . The correction is applied to the problem of calibrating p-values by transforming them to estimated Bayes factors. The corresponding calibration in the most common case is simply sqrt(n)/exp((1−1/n)*qchisq(1−p,df=1)/2) in R syntax, where the p-value is from a likelihood-ratio test. That intersects the class of betting scores called e-values and, more specifically, admissible calibrators. While all admissible calibrators neither overstate nor understate the evidence against the null hypothesis, previous admissible calibrators are not model-selection consistent since they do not increasingly favor the null hypothesis when it is true. The proposed calibrator is consistent under general conditions, for its corrected BIC is asymptotically equivalent to the BIC.

Suggested Citation

  • David R. Bickel, 2025. "A small-sample Bayesian information criterion that does not overstate the evidence, with an application to calibrating p-values from likelihood-ratio tests," Statistical Papers, Springer, vol. 66(3), pages 1-17, April.
  • Handle: RePEc:spr:stpapr:v:66:y:2025:i:3:d:10.1007_s00362-025-01682-1
    DOI: 10.1007/s00362-025-01682-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-025-01682-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00362-025-01682-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sander Greenland, 2019. "Valid P-Values Behave Exactly as They Should: Some Misleading Criticisms of P-Values and Their Resolution With S-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 106-114, March.
    2. Daniel J. Benjamin & James O. Berger, 2019. "Three Recommendations for Improving the Use of p-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 186-191, March.
    3. McQuarrie, Allan D., 1999. "A small-sample correction for the Schwarz SIC model selection criterion," Statistics & Probability Letters, Elsevier, vol. 44(1), pages 79-86, August.
    4. Sander Greenland, 2023. "Divergence versus decision P‐values: A distinction worth making in theory and keeping in practice: Or, how divergence P‐values measure evidence even when decision P‐values do not," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(1), pages 54-88, March.
    5. David R. Bickel, 2021. "The sufficiency of the evidence, the relevancy of the evidence, and quantifying both with a single number," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(4), pages 1157-1174, October.
    6. Thomas A. Severini, 2007. "Integrated likelihood functions for non-Bayesian inference," Biometrika, Biometrika Trust, vol. 94(3), pages 529-542.
    7. Spencer Hansen & Ken Rice, 2023. "Coherent Tests for Interval Null Hypotheses," The American Statistician, Taylor & Francis Journals, vol. 77(1), pages 20-28, January.
    8. David R. Bickel, 2023. "Statistical evidence and surprise unified under possibility theory," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(3), pages 923-928, September.
    9. Glenn Shafer, 2021. "Author's reply to the Discussion of ‘Testing by betting: A strategy for statistical and scientific communication’ by Glenn Shafer," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(2), pages 466-478, April.
    10. Alexandre Galvão Patriota, 2022. "A measure of evidence based on the likelihood-ratio statistics," Statistical Papers, Springer, vol. 63(6), pages 1931-1951, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David R. Bickel, 2024. "The p-value interpreted as the posterior probability of explaining the data: Applications to multiple testing and to restricted parameter spaces," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 86(1), pages 464-493, February.
    2. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    3. Eleni Verykouki & Christos T. Nakas, 2023. "Adaptations on the Use of p -Values for Statistical Inference: An Interpretation of Messages from Recent Public Discussions," Stats, MDPI, vol. 6(2), pages 1-13, April.
    4. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.
    5. Sondang Marsinta Uli Panggabean & Mahjus Ekananda & Beta Yulianita Gitaharie & Leslie Djuranovik, 2025. "Export proceeds repatriation policies: A shield against exchange rate volatility in emerging markets?," Papers 2506.09168, arXiv.org.
    6. Bibhas Chakraborty, 2020. "Statistical Remedies for Medical Researchers," International Statistical Review, International Statistical Institute, vol. 88(3), pages 802-804, December.
    7. Giuliana Cortese & Nicola Sartori, 2016. "Integrated likelihoods in parametric survival models for highly clustered censored data," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 22(3), pages 382-404, July.
    8. repec:hum:wpaper:sfb649dp2013-028 is not listed on IDEAS
    9. Carlos A. Medel, 2015. "Probabilidad Clásica de Sobreajuste con Criterios de Información: Estimaciones con Series Macroeconómicas Chilenas," Revista de Analisis Economico – Economic Analysis Review, Universidad Alberto Hurtado/School of Economics and Business, vol. 30(1), pages 57-72, Abril.
    10. Schumann, Martin & Severini, Thomas A. & Tripathi, Gautam, 2021. "Integrated likelihood based inference for nonlinear panel data models with unobserved effects," Journal of Econometrics, Elsevier, vol. 223(1), pages 73-95.
    11. Härdle, Wolfgang Karl & Huang, Li-shan, 2013. "Analysis of deviance in generalized partial linear models," SFB 649 Discussion Papers 2013-028, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    12. Pakel, Cavit, 2019. "Bias reduction in nonlinear and dynamic panels in the presence of cross-section dependence," Journal of Econometrics, Elsevier, vol. 213(2), pages 459-492.
    13. Michał Marcin Kobierecki & Michał Pierzgalski, 2022. "Sports Mega-Events and Economic Growth: A Synthetic Control Approach," Journal of Sports Economics, , vol. 23(5), pages 567-597, June.
    14. V. Filimonov & G. Demos & D. Sornette, 2017. "Modified profile likelihood inference and interval forecast of the burst of financial bubbles," Quantitative Finance, Taylor & Francis Journals, vol. 17(8), pages 1167-1186, August.
    15. H. V. Kulkarni & S. M. Patil, 2021. "Uniformly implementable small sample integrated likelihood ratio test for one-way and two-way ANOVA under heteroscedasticity and normality," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 105(2), pages 273-305, June.
    16. Michal Lehnert & Mark De Ste Croix & Amr Zaatar & Patrycja Lipinska & Petr Stastny, 2020. "Effect of a Simulated Match on Lower Limb Neuromuscular Performance in Youth Footballers—A Two Year Longitudinal Study," IJERPH, MDPI, vol. 17(22), pages 1-17, November.
    17. Thomas A. Severini, 2023. "Integrated likelihood inference in multinomial distributions," METRON, Springer;Sapienza Università di Roma, vol. 81(2), pages 131-142, August.
    18. Qichang Xie & Yingkun Yan & Xu Wang, 2023. "Assessing the role of foreign direct investment in environmental sustainability: a spatial semiparametric panel approach," Economic Change and Restructuring, Springer, vol. 56(2), pages 1263-1295, April.
    19. Ventura, Laura & Racugno, Walter, 2012. "On interval and point estimators based on a penalization of the modified profile likelihood," Statistics & Probability Letters, Elsevier, vol. 82(7), pages 1285-1289.
    20. Cheng, Yuanyuan, 2023. "A method of 3R to evaluate the correlation and predictive value of variables," OSF Preprints c79tu, Center for Open Science.
    21. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:66:y:2025:i:3:d:10.1007_s00362-025-01682-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.