IDEAS home Printed from https://ideas.repec.org/a/spr/sankha/v86y2024i1d10.1007_s13171-023-00328-4.html
   My bibliography  Save this article

The p-value interpreted as the posterior probability of explaining the data: Applications to multiple testing and to restricted parameter spaces

Author

Listed:
  • David R. Bickel

    (University of North Carolina at Greensboro, The Graduate School)

Abstract

Failures to replicate the results of scientific studies are often attributed to misinterpretations of the p-value. The p-value may be interpreted as an approximate posterior probability, not that the null hypothesis is true but rather that it explains the data as well as the data-generating distribution. That posterior probability modifies the p-value in the following two broad areas of application, leading to new methods of hypothesis testing and effect size estimation. First, when corrected for multiple comparisons, the posterior probability that the null hypothesis adequately explains the data overcomes both the conservative bias of corrected p-values and the anti-conservative bias of commonly used false discovery rate methods. Second, the posterior probability that the null hypothesis adequately explains the data, conditional on a parameter restriction, transforms the p-value in such a way as to overcome difficulties in restricted parameter spaces.

Suggested Citation

  • David R. Bickel, 2024. "The p-value interpreted as the posterior probability of explaining the data: Applications to multiple testing and to restricted parameter spaces," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 86(1), pages 464-493, February.
    • Handle: RePEc:spr:sankha:v:86:y:2024:i:1:d:10.1007_s13171-023-00328-4
    DOI: 10.1007/s13171-023-00328-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s13171-023-00328-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s13171-023-00328-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sander Greenland, 2019. "Valid P-Values Behave Exactly as They Should: Some Misleading Criticisms of P-Values and Their Resolution With S-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 106-114, March.
    2. Daniel J. Benjamin & James O. Berger, 2019. "Three Recommendations for Improving the Use of p-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 186-191, March.
    3. Brendan Kline, 2022. "Bayes Factors Based on p-Values and Sets of Priors With Restricted Strength," The American Statistician, Taylor & Francis Journals, vol. 76(3), pages 203-213, July.
    4. Frank G Ball, 2002. "Empty confidence sets for epidemics, branching processes and Brownian motion," Biometrika, Biometrika Trust, vol. 89(1), pages 211-224, March.
    5. Wang, Hsiuying, 2007. "Modified p-values for one-sided testing in restricted parameter spaces," Statistics & Probability Letters, Elsevier, vol. 77(6), pages 625-631, March.
    6. David R. Bickel, 2021. "Null Hypothesis Significance Testing Defended and Calibrated by Bayesian Model Checking," The American Statistician, Taylor & Francis Journals, vol. 75(3), pages 249-255, July.
    7. David R. Bickel, 2021. "The sufficiency of the evidence, the relevancy of the evidence, and quantifying both with a single number," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(4), pages 1157-1174, October.
    8. Ronald L. Wasserstein & Allen L. Schirm & Nicole A. Lazar, 2019. "Moving to a World Beyond “p," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 1-19, March.
    9. Haolun Shi & Guosheng Yin, 2021. "Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests," The American Statistician, Taylor & Francis Journals, vol. 75(3), pages 265-275, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    2. David R. Bickel, 2025. "A small-sample Bayesian information criterion that does not overstate the evidence, with an application to calibrating p-values from likelihood-ratio tests," Statistical Papers, Springer, vol. 66(3), pages 1-17, April.
    3. Eleni Verykouki & Christos T. Nakas, 2023. "Adaptations on the Use of p -Values for Statistical Inference: An Interpretation of Messages from Recent Public Discussions," Stats, MDPI, vol. 6(2), pages 1-13, April.
    4. Riko Kelter, 2024. "The Case of the Jeffreys-Lindley-paradox as a Bayes-frequentist Compromise: A Perspective Based on the Rao-Lovric-Theorem," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 86(1), pages 337-363, February.
    5. Pütz, Peter & Kramer-Sunderbrink, Arne & Dreher, Robin Tim & Hoffmann, Leona & Werner, Robin, 2022. "A Proposed Hybrid Effect Size Plus p-Value Criterion. A Comment on Goodman et al. (The American Statistician, 2019)," Journal of Comments and Replications in Economics (JCRE), ZBW - Leibniz Information Centre for Economics, vol. 1, pages 1-15.
    6. Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
    7. David R. Bickel, 2023. "Statistical evidence and surprise unified under possibility theory," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(3), pages 923-928, September.
    8. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.
    9. Bibhas Chakraborty, 2020. "Statistical Remedies for Medical Researchers," International Statistical Review, International Statistical Institute, vol. 88(3), pages 802-804, December.
    10. Maria-Lavinia FLOREA & Anca BORZA, 2019. "Individual Strategies For Achieving Work-Life Balance €“ A Case Study On Romanian Workers," Proceedings of the INTERNATIONAL MANAGEMENT CONFERENCE, Faculty of Management, Academy of Economic Studies, Bucharest, Romania, vol. 13(1), pages 845-855, November.
    11. Grüner Sven, 2020. "Sample Size Calculation in Economic Experiments," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 240(6), pages 791-823, December.
    12. Michał Marcin Kobierecki & Michał Pierzgalski, 2022. "Sports Mega-Events and Economic Growth: A Synthetic Control Approach," Journal of Sports Economics, , vol. 23(5), pages 567-597, June.
    13. Michaelides, Michael, 2021. "Large sample size bias in empirical finance," Finance Research Letters, Elsevier, vol. 41(C).
    14. Kelter, Riko, 2022. "Power analysis and type I and type II error rates of Bayesian nonparametric two-sample tests for location-shifts based on the Bayes factor under Cauchy priors," Computational Statistics & Data Analysis, Elsevier, vol. 165(C).
    15. John Cawley & Alex Susskind & Barton Willage, 2020. "The Impact of Information Disclosure on Consumer Behavior: Evidence from a Randomized Field Experiment of Calorie Labels on Restaurant Menus," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(4), pages 1020-1042, September.
    16. Abbas, Sadia & Adapa, Sujana & Sheridan, Alison & Azeem, Muhammad Masood, 2022. "Informal competition and firm level innovation in South Asia: The moderating role of innovation time off and R&D intensity," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    17. Michal Lehnert & Mark De Ste Croix & Amr Zaatar & Patrycja Lipinska & Petr Stastny, 2020. "Effect of a Simulated Match on Lower Limb Neuromuscular Performance in Youth Footballers—A Two Year Longitudinal Study," IJERPH, MDPI, vol. 17(22), pages 1-17, November.
    18. Pete Leasure & Hunter M. Boehme, 2024. "An Experimental Investigation Examining the Impact of Medical Association Statements about Drug Addiction on Perceptions of Criminal Culpability and Punishment," Social Sciences, MDPI, vol. 13(7), pages 1-8, July.
    19. Matthias Breuer & Ed Dehaan, 2024. "Using and Interpreting Fixed Effects Models," Journal of Accounting Research, Wiley Blackwell, vol. 62(4), pages 1183-1226, September.
    20. Simon Berset & Martin Huber & Mark Schelker, 2023. "The fiscal response to revenue shocks," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 30(3), pages 814-848, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:sankha:v:86:y:2024:i:1:d:10.1007_s13171-023-00328-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.