IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/t93cg.html

Toward More Transparency in Statistical Practice

Author

Listed:
  • Wagenmakers, Eric-Jan

    (University of Amsterdam)

  • Sarafoglou, Alexandra

    (University of Amsterdam)

  • Aarts, Sil Dr.

    (Maastricht University)

  • Albers, Casper J

    (University of Groningen)

  • Algermissen, Johannes

    (Radboud University Nijmegen)

  • Bahník, Štěpán

    (University of Economics, Prague)

  • van Dongen, Noah N'Djaye Nikolai
  • Hoekstra, Rink
  • Moreau, David
  • van Ravenzwaaij, Don

    (University of Groningen)

Abstract

We explore the promise of statistical reform by starting from the assumption that most researchers would endorse Merton's ethos of science as reflected in the four norms of communalism, universalism, disinterestedness, and organized skepticism. Translated to data analysis, these norms imply a need for transparency, a fair acknowledgement of uncertainty, and openness to alternative interpretations. We discuss seven statistical procedures, both old and new, that we believe can positively impact statistical practice in the social and behavioral sciences.

Suggested Citation

  • Wagenmakers, Eric-Jan & Sarafoglou, Alexandra & Aarts, Sil Dr. & Albers, Casper J & Algermissen, Johannes & Bahník, Štěpán & van Dongen, Noah N'Djaye Nikolai & Hoekstra, Rink & Moreau, David & van Rav, 2021. "Toward More Transparency in Statistical Practice," MetaArXiv t93cg, Center for Open Science.
    • Handle: RePEc:osf:metaar:t93cg
    DOI: 10.31219/osf.io/t93cg
    as

    Download full text from publisher

    File URL: https://osf.io/download/603e494967386c031361e531/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/t93cg?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Darren B Taichman & Peush Sahni & Anja Pinborg & Larry Peiperl & Christine Laine & Astrid James & Sung-Tae Hong & Abraham Haileamlak & Laragh Gollogly & Fiona Godlee & Frank A Frizelle & Fernando Flor, 2017. "Data Sharing Statements for Clinical Trials: A Requirement of the International Committee of Medical Journal Editors," PLOS Medicine, Public Library of Science, vol. 14(6), pages 1-3, June.
    2. Noah N. N. van Dongen & Johnny B. van Doorn & Quentin F. Gronau & Don van Ravenzwaaij & Rink Hoekstra & Matthias N. Haucke & Daniel Lakens & Christian Hennig & Richard D. Morey & Saskia Homer & Andrew, 2019. "Multiple Perspectives on Inference for Two Simple Statistical Scenarios," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 328-339, March.
    3. Gelman A. & Pasarica C. & Dodhia R., 2002. "Lets Practice What We Preach: Turning Tables into Graphs," The American Statistician, American Statistical Association, vol. 56, pages 121-130, May.
    4. Susan Athey & Guido W. Imbens, 2019. "Machine Learning Methods That Economists Should Know About," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 685-725, August.
    5. Athey, Susan & Imbens, Guido W., 2019. "Machine Learning Methods Economists Should Know About," Research Papers 3776, Stanford University, Graduate School of Business.
    6. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1208-1214, November.
    7. Blakeley B. McShane & David Gal & Andrew Gelman & Christian Robert & Jennifer L. Tackett, 2019. "Abandon Statistical Significance," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 235-245, March.
    8. Leamer, Edward E, 1985. "Sensitivity Analyses Would Help," American Economic Review, American Economic Association, vol. 75(3), pages 308-313, June.
    9. Levine, Ross & Renelt, David, 1992. "A Sensitivity Analysis of Cross-Country Growth Regressions," American Economic Review, American Economic Association, vol. 82(4), pages 942-963, September.
    10. Jonah Gabry & Daniel Simpson & Aki Vehtari & Michael Betancourt & Andrew Gelman, 2019. "Visualization in Bayesian workflow," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 182(2), pages 389-402, February.
    11. Tracey L Weissgerber & Natasa M Milic & Stacey J Winham & Vesna D Garovic, 2015. "Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm," PLOS Biology, Public Library of Science, vol. 13(4), pages 1-10, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bennett, Davara L. & Webb, Calum J.R. & Mason, Kate E. & Schlüter, Daniela K. & Fahy, Katie & Alexiou, Alexandros & Wickham, Sophie & Barr, Ben & Taylor-Robinson, David, 2021. "Funding for preventative Children’s Services and rates of children becoming looked after: A natural experiment using longitudinal area-level data in England," Children and Youth Services Review, Elsevier, vol. 131(C).
    2. Sacker, Amanda & Lacey, Rebecca E. & Maughan, Barbara & Murray, Emily T., 2022. "Out-of-home care in childhood and socio-economic functioning in adulthood: ONS Longitudinal study 1971–2011," Children and Youth Services Review, Elsevier, vol. 132(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eric-Jan Wagenmakers & Alexandra Sarafoglou & Sil Aarts & Casper Albers & Johannes Algermissen & Štěpán Bahník & Noah Dongen & Rink Hoekstra & David Moreau & Don Ravenzwaaij & Aljaž Sluga & Franziska , 2021. "Seven steps toward more transparency in statistical practice," Nature Human Behaviour, Nature, vol. 5(11), pages 1473-1480, November.
    2. repec:osf:metaar:t93cg_v1 is not listed on IDEAS
    3. repec:osf:socarx:tjkcy_v1 is not listed on IDEAS
    4. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    5. Gabriel Okasa & Kenneth A. Younge, 2022. "Sample Fit Reliability," Papers 2209.06631, arXiv.org.
    6. Sophie-Charlotte Klose & Johannes Lederer, 2020. "A Pipeline for Variable Selection and False Discovery Rate Control With an Application in Labor Economics," Papers 2006.12296, arXiv.org, revised Jun 2020.
    7. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP72/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    8. Labib Shami & Teddy Lazebnik, 2024. "Implementing Machine Learning Methods in Estimating the Size of the Non-observed Economy," Computational Economics, Springer;Society for Computational Economics, vol. 63(4), pages 1459-1476, April.
    9. Rok Spruk & Mitja Kovac, 2018. "Inefficient Growth," Review of Economics and Institutions, Università di Perugia, vol. 9(2).
    10. Hurmeranta, Risto & Lyytikäinen, Teemu, 2025. "Nominal Loss Aversion in the Housing Market and Household Mobility," Working Papers 178, VATT Institute for Economic Research.
    11. Chen, Ruoyu & Jiang, Hanchen & Quintero, Luis E., 2023. "Measuring the value of rent stabilization and understanding its implications for racial inequality: Evidence from New York City," Regional Science and Urban Economics, Elsevier, vol. 103(C).
    12. Dang, Hai-Anh & Carleto, Gero & Gourlay, Sydney & Abanokova, Kseniya, 2023. "Addressing Soil Quality Data Gaps with Imputation: Evidence from Ethiopia and Uganda," 2023 Annual Meeting, July 23-25, Washington D.C. 335648, Agricultural and Applied Economics Association.
    13. R Burger & S du Plessis, 2011. "Examining the Robustness of Competing Explanations of Slow Growth in African Countries," Studies in Economics and Econometrics, Taylor & Francis Journals, vol. 35(3), pages 21-47, December.
    14. Dangxing Chen & Luyao Zhang, 2023. "Monotonicity for AI ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance," Papers 2301.07060, arXiv.org.
    15. Ballestar, María Teresa & Mir, Miguel Cuerdo & Pedrera, Luis Miguel Doncel & Sainz, Jorge, 2024. "Effectiveness of tutoring at school: A machine learning evaluation," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    16. Carmen Fernandez & Eduardo Ley & Mark F. J. Steel, 2001. "Model uncertainty in cross-country growth regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 16(5), pages 563-576.
    17. Combes, Pierre-Philippe & Gobillon, Laurent & Zylberberg, Yanos, 2022. "Urban economics in a historical perspective: Recovering data with machine learning," Regional Science and Urban Economics, Elsevier, vol. 94(C).
    18. Martin Gassebner & Jerg Gutmann & Stefan Voigt, 2016. "When to expect a coup d’état? An extreme bounds analysis of coup determinants," Public Choice, Springer, vol. 169(3), pages 293-313, December.
    19. Barzin,Samira & Avner,Paolo & Maruyama Rentschler,Jun Erik & O’Clery,Neave, 2022. "Where Are All the Jobs ? A Machine Learning Approach for High Resolution Urban Employment Prediction inDeveloping Countries," Policy Research Working Paper Series 9979, The World Bank.
    20. Arenas, Andreu & Calsamiglia, Caterina, 2022. "Gender Differences in High-Stakes Performance and College Admission Policies," IZA Discussion Papers 15550, Institute of Labor Economics (IZA).
    21. Tsang, Andrew, 2021. "Uncovering Heterogeneous Regional Impacts of Chinese Monetary Policy," MPRA Paper 110703, University Library of Munich, Germany.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:t93cg. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.