IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v81y2016i1p1-15.html
   My bibliography  Save this article

Playing with Data—Or How to Discourage Questionable Research Practices and Stimulate Researchers to Do Things Right

Author

Listed:
  • Klaas Sijtsma

Abstract

Recent fraud cases in psychological and medical research have emphasized the need to pay attention to Questionable Research Practices (QRPs). Deliberate or not, QRPs usually have a deteriorating effect on the quality and the credibility of research results. QRPs must be revealed but prevention of QRPs is more important than detection. I suggest two policy measures that I expect to be effective in improving the quality of psychological research. First, the research data and the research materials should be made publicly available so as to allow verification. Second, researchers should more readily consider consulting a methodologist or a statistician. These two measures are simple but run against common practice to keep data to oneself and overestimate one’s methodological and statistical skills, thus allowing secrecy and errors to enter research practice. Copyright The Psychometric Society 2016

Suggested Citation

  • Klaas Sijtsma, 2016. "Playing with Data—Or How to Discourage Questionable Research Practices and Stimulate Researchers to Do Things Right," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 1-15, March.
    • Handle: RePEc:spr:psycho:v:81:y:2016:i:1:p:1-15
    DOI: 10.1007/s11336-015-9446-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11336-015-9446-0
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11336-015-9446-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
    2. Daniele Fanelli, 2013. "Redefine misconduct as distorted reporting," Nature, Nature, vol. 494(7436), pages 149-149, February.
    3. Klaas Sijtsma, 2012. "Future of Psychometrics: Ask What Psychometrics Can Do for Psychology," Psychometrika, Springer;The Psychometric Society, vol. 77(1), pages 4-20, January.
    4. Jelte M Wicherts & Marjan Bakker & Dylan Molenaar, 2011. "Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results," PLOS ONE, Public Library of Science, vol. 6(11), pages 1-7, November.
    5. Daniele Fanelli, 2009. "How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data," PLOS ONE, Public Library of Science, vol. 4(5), pages 1-11, May.
    6. Lee Cronbach, 1954. "Report on a psychometric mission to clinicia," Psychometrika, Springer;The Psychometric Society, vol. 19(4), pages 263-270, December.
    7. Denny Borsboom, 2006. "The attack of the psychometricians," Psychometrika, Springer;The Psychometric Society, vol. 71(3), pages 425-440, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anand Krishna & Sebastian M Peter, 2018. "Questionable research practices in student final theses – Prevalence, attitudes, and the role of the supervisor’s perceived attitudes," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-24, August.
    2. Herman Aguinis & Wayne F. Cascio & Ravi S. Ramani, 2017. "Science’s reproducibility and replicability crisis: International business is not immune," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 48(6), pages 653-663, August.
    3. Hengky Latan & Charbel Jose Chiappetta Jabbour & Ana Beatriz Lopes de Sousa Jabbour & Murad Ali, 2023. "Crossing the Red Line? Empirical Evidence and Useful Recommendations on Questionable Research Practices among Business Scholars," Journal of Business Ethics, Springer, vol. 184(3), pages 549-569, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klaas Sijtsma & Coosje Veldkamp & Jelte Wicherts, 2016. "Improving the Conduct and Reporting of Statistical Analysis in Psychology," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 33-38, March.
    2. Aguinis, Herman & Banks, George C. & Rogelberg, Steven G. & Cascio, Wayne F., 2020. "Actionable recommendations for narrowing the science-practice gap in open science," Organizational Behavior and Human Decision Processes, Elsevier, vol. 158(C), pages 27-35.
    3. Frederique Bordignon, 2020. "Self-correction of science: a comparative study of negative citations and post-publication peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1225-1239, August.
    4. Brian Fabo & Martina Jancokova & Elisabeth Kempf & Lubos Pastor, 2020. "Fifty Shades of QE: Conflicts of Interest in Economic Research," Working Papers 2020-128, Becker Friedman Institute for Research In Economics.
    5. Stephan B Bruns & John P A Ioannidis, 2016. "p-Curve and p-Hacking in Observational Research," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-13, February.
    6. David Spiegelhalter, 2017. "Trust in numbers," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 948-965, October.
    7. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    8. Klaas Sijtsma, 2016. "Playing with Data—Or How to Discourage Questionable Research Practices and Stimulate Researchers to Do Things Right," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 1-15, March.
    9. Fabo, Brian & Jančoková, Martina & Kempf, Elisabeth & Pástor, Ľuboš, 2021. "Fifty shades of QE: Comparing findings of central bankers and academics," Journal of Monetary Economics, Elsevier, vol. 120(C), pages 1-20.
    10. Koessler, Ann-Kathrin & Page, Lionel & Dulleck, Uwe, 2015. "Promoting pro-social behavior with public statements of good intent," MPRA Paper 80072, University Library of Munich, Germany, revised 24 May 2017.
    11. Matteo Colombo & Georgi Duev & Michèle B Nuijten & Jan Sprenger, 2018. "Statistical reporting inconsistencies in experimental philosophy," PLOS ONE, Public Library of Science, vol. 13(4), pages 1-12, April.
    12. David Pontille & Didier Torny, 2013. "Behind the scenes of scientific articles: defining categories of fraud and regulating cases," CSI Working Papers Series 031, Centre de Sociologie de l'Innovation (CSI), Mines ParisTech.
    13. Ádám Kun, 2018. "Publish and Who Should Perish: You or Science?," Publications, MDPI, vol. 6(2), pages 1-16, April.
    14. Gall, Thomas & Maniadis, Zacharias, 2019. "Evaluating solutions to the problem of false positives," Research Policy, Elsevier, vol. 48(2), pages 506-515.
    15. Irwin Waldman & Scott Lilienfeld, 2016. "Thinking About Data, Research Methods, and Statistical Analyses: Commentary on Sijtsma’s (2014) “Playing with Data”," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 16-26, March.
    16. Herman Aguinis & Wayne F. Cascio & Ravi S. Ramani, 2017. "Science’s reproducibility and replicability crisis: International business is not immune," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 48(6), pages 653-663, August.
    17. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    18. Michał Krawczyk, 2015. "The Search for Significance: A Few Peculiarities in the Distribution of P Values in Experimental Psychology Literature," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-19, June.
    19. Kraft-Todd, Gordon T. & Rand, David G., 2021. "Practice what you preach: Credibility-enhancing displays and the growth of open science," Organizational Behavior and Human Decision Processes, Elsevier, vol. 164(C), pages 1-10.
    20. S. P. J. M. Horbach & W. Halffman, 2019. "The ability of different peer review procedures to flag problematic publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 339-373, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:81:y:2016:i:1:p:1-15. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.