IDEAS home Printed from https://ideas.repec.org/a/spr/jqecon/v20y2022i1d10.1007_s40953-022-00312-z.html
   My bibliography  Save this article

Frequentist Model-based Statistical Induction and the Replication Crisis

Author

Listed:
  • Aris Spanos

    (Virginia Tech)

Abstract

The prevailing view in the current replication crisis literature is that the non-replicability of published empirical studies (a) confirms their untrustworthiness, and (b) the primary source of that is the abuse of frequentist testing, in general, and the p-value in particular. The main objective of the paper is to challenge both of these claims and make a case that (a) non-replicability does not necessarily imply untrustworthiness and (b) the abuses of frequentist testing are only symptomatic of a much broader problem relating to the uninformed and recipe-like implementation of statistical modeling and inference that contributes significantly to untrustworthy evidence. It is argued that the crucial contributors to the untrustworthiness relate (directly or indirectly) to the inadequate understanding and implementation of the stipulations required for model-based statistical induction to give rise to trustworthy evidence. It is argued that these preconditions relate to securing reliable ‘learning from data’ about phenomena of interest and pertain to the nature, origin, and justification of genuine empirical knowledge, as opposed to beliefs, conjectures, and opinions.

Suggested Citation

  • Aris Spanos, 2022. "Frequentist Model-based Statistical Induction and the Replication Crisis," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 20(1), pages 133-159, September.
    • Handle: RePEc:spr:jqecon:v:20:y:2022:i:1:d:10.1007_s40953-022-00312-z
    DOI: 10.1007/s40953-022-00312-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40953-022-00312-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40953-022-00312-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sundberg,Rolf, 2019. "Statistical Modelling by Exponential Families," Cambridge Books, Cambridge University Press, number 9781108701112.
    2. Spanos, Aris, 2010. "Akaike-type criteria and the reliability of inference: Model selection versus statistical model specification," Journal of Econometrics, Elsevier, vol. 158(2), pages 204-220, October.
    3. Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
      • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
    4. Spanos,Aris, 1986. "Statistical Foundations of Econometric Modelling," Cambridge Books, Cambridge University Press, number 9780521269124.
    5. Spanos,Aris, 2019. "Probability Theory and Statistical Inference," Cambridge Books, Cambridge University Press, number 9781316636374.
    6. Sundberg,Rolf, 2019. "Statistical Modelling by Exponential Families," Cambridge Books, Cambridge University Press, number 9781108476591.
    7. Aris Spanos & Anya McGuirk, 2001. "The Model Specification Problem from a Probabilistic Reduction Perspective," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 83(5), pages 1168-1176.
    8. Aris Spanos, 2018. "Mis†Specification Testing In Retrospect," Journal of Economic Surveys, Wiley Blackwell, vol. 32(2), pages 541-577, April.
    9. Aris Spanos, 2006. "Revisiting the omitted variables argument: Substantive vs. statistical adequacy," Journal of Economic Methodology, Taylor & Francis Journals, vol. 13(2), pages 179-218.
    10. Jeffrey T. Leek & Roger D. Peng, 2015. "Statistics: P values are just the tip of the iceberg," Nature, Nature, vol. 520(7549), pages 612-612, April.
    11. Spanos, Aris, 1990. "The simultaneous-equations model revisited : Statistical adequacy and identification," Journal of Econometrics, Elsevier, vol. 44(1-2), pages 87-105.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aris Spanos, 2016. "Transforming structural econometrics: substantive vs. statistical premises of inference," Review of Political Economy, Taylor & Francis Journals, vol. 28(3), pages 426-437, July.
    2. Spanos, Aris, 2010. "Statistical adequacy and the trustworthiness of empirical evidence: Statistical vs. substantive information," Economic Modelling, Elsevier, vol. 27(6), pages 1436-1452, November.
    3. Niraj Poudyal & Aris Spanos, 2022. "Model Validation and DSGE Modeling," Econometrics, MDPI, vol. 10(2), pages 1-25, April.
    4. Spanos, Aris, 2010. "Akaike-type criteria and the reliability of inference: Model selection versus statistical model specification," Journal of Econometrics, Elsevier, vol. 158(2), pages 204-220, October.
    5. Aris Spanos, 2018. "Mis†Specification Testing In Retrospect," Journal of Economic Surveys, Wiley Blackwell, vol. 32(2), pages 541-577, April.
    6. Aris Spanos, 2021. "Yule–Simpson’s paradox: the probabilistic versus the empirical conundrum," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(2), pages 605-635, June.
    7. Aris Spanos, 2022. "Statistical modeling and inference in the era of Data Science and Graphical Causal modeling," Journal of Economic Surveys, Wiley Blackwell, vol. 36(5), pages 1251-1287, December.
    8. Aris Spanos, 2009. "Statistical Misspecification and the Reliability of Inference: The Simple T-Test in the Presence of Markov Dependence," Korean Economic Review, Korean Economic Association, vol. 25, pages 165-213.
    9. Spanos, Aris, 2009. "The Pre-Eminence of Theory versus the European CVAR Perspective in Macroeconometric Modeling," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 3, pages 1-14.
    10. Anya McGuirk & Aris Spanos, 2009. "Revisiting Error‐Autocorrelation Correction: Common Factor Restrictions and Granger Non‐Causality," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(2), pages 273-294, April.
    11. Michaelides, Michael & Spanos, Aris, 2020. "On modeling heterogeneity in linear models using trend polynomials," Economic Modelling, Elsevier, vol. 85(C), pages 74-86.
    12. P. Dorian Owen, 2017. "Evaluating Ingenious Instruments for Fundamental Determinants of Long-Run Economic Growth and Development," Econometrics, MDPI, vol. 5(3), pages 1-33, September.
    13. Owen, P. Dorian, 2018. "Replication to assess statistical adequacy," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-16.
    14. Park, Jaewoo & Jin, Ick Hoon & Schweinberger, Michael, 2022. "Bayesian model selection for high-dimensional Ising models, with applications to educational data," Computational Statistics & Data Analysis, Elsevier, vol. 165(C).
    15. Whiteley, Nick, 2021. "Dimension-free Wasserstein contraction of nonlinear filters," Stochastic Processes and their Applications, Elsevier, vol. 135(C), pages 31-50.
    16. Aris Spanos & Niki Papadopoulou, 2013. "A Small Macroeconometric Model for the Cyprus Economy," Working Papers 2013-2, Central Bank of Cyprus.
    17. Spanos, Aris, 1989. "On Rereading Haavelmo: A Retrospective View of Econometric Modeling," Econometric Theory, Cambridge University Press, vol. 5(3), pages 405-429, December.
    18. Jaime Ortiz, 1999. "The role of interest groups in agricultural policy design: Chile 1960-1988," Journal of International Development, John Wiley & Sons, Ltd., vol. 11(2), pages 241-258.
    19. Vassilios Bazinas & Bent Nielsen, 2022. "Causal Transmission in Reduced-Form Models," Econometrics, MDPI, vol. 10(2), pages 1-25, March.
    20. Michael Schweinberger, 2022. "Discussion to: Bayesian graphical models for modern biological applications by Y. Ni, V. Baladandayuthapani, M. Vannucci and F.C. Stingo," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 31(2), pages 253-260, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jqecon:v:20:y:2022:i:1:d:10.1007_s40953-022-00312-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.