IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v51y2022i2p499-526.html
   My bibliography  Save this article

Transparency and Replication in Cross-national Survey Research: Identification of Problems and Possible Solutions

Author

Listed:
  • Elena Damian
  • Bart Meuleman
  • Wim van Oorschot

Abstract

In this article, we examine whether cross-national studies disclose enough information for independent researchers to evaluate the validity and reliability of the findings (evaluation transparency) or to perform a direct replication (replicability transparency). The first contribution is theoretical. We develop a heuristic theoretical model including the actors, factors, and processes that influence the transparency of cross-national studies and provide an overview of the measures currently taken to improve research transparency. The second contribution is empirical, in which we analyze the level of transparency in published cross-national studies. Specifically, using a random sample of 305 comparative studies published in one of 29 peer-reviewed social sciences journals (from 1986 to 2016), we show that, even though all the articles include some methodological information, the great majority lack sufficient information for evaluation and replication. Lastly, we develop and propose a set of transparency guidelines tailored for reporting cross-national survey research.

Suggested Citation

  • Elena Damian & Bart Meuleman & Wim van Oorschot, 2022. "Transparency and Replication in Cross-national Survey Research: Identification of Problems and Possible Solutions," Sociological Methods & Research, , vol. 51(2), pages 499-526, May.
    • Handle: RePEc:sae:somere:v:51:y:2022:i:2:p:499-526
    DOI: 10.1177/0049124119882452
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124119882452
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124119882452?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daniele Fanelli, 2009. "How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data," PLOS ONE, Public Library of Science, vol. 4(5), pages 1-11, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Moustafa, Khaled, 2018. "Don't fall in common science pitfall!," FrenXiv ycjha, Center for Open Science.
    2. Love, Peter E.D. & Ika, Lavagnon A. & Ahiaga-Dagbui, Dominic D., 2019. "On de-bunking ‘fake news’ in a post truth era: Why does the Planning Fallacy explanation for cost overruns fall short?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 126(C), pages 397-408.
    3. Jeremy Hall & Ben R. Martin, 2019. "Towards a Taxonomy of Academic Misconduct: The Case of Business School Research," SPRU Working Paper Series 2019-02, SPRU - Science Policy Research Unit, University of Sussex Business School.
    4. Kartal, Melis & Tremewan, James, 2018. "An offer you can refuse: The effect of transparency with endogenous conflict of interest," Journal of Public Economics, Elsevier, vol. 161(C), pages 44-55.
    5. Robert J Warren II & Joshua R King & Charlene Tarsa & Brian Haas & Jeremy Henderson, 2017. "A systematic review of context bias in invasion biology," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-12, August.
    6. Jasper Brinkerink, 2023. "When Shooting for the Stars Becomes Aiming for Asterisks: P-Hacking in Family Business Research," Entrepreneurship Theory and Practice, , vol. 47(2), pages 304-343, March.
    7. Frederique Bordignon, 2020. "Self-correction of science: a comparative study of negative citations and post-publication peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1225-1239, August.
    8. Hensel, Przemysław G., 2019. "Supporting replication research in management journals: Qualitative analysis of editorials published between 1970 and 2015," European Management Journal, Elsevier, vol. 37(1), pages 45-57.
    9. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    10. Gary Charness & David Masclet & Marie Claire Villeval, 2014. "The Dark Side of Competition for Status," Management Science, INFORMS, vol. 60(1), pages 38-55, January.
    11. Brian Fabo & Martina Jancokova & Elisabeth Kempf & Lubos Pastor, 2020. "Fifty Shades of QE: Conflicts of Interest in Economic Research," Working and Discussion Papers WP 5/2020, Research Department, National Bank of Slovakia.
    12. Bruce B. Svare, 2020. "A Cautionary Tale for Psychology and Higher Education in Asia: Following Western Practices of Incentivising Scholarship May Have Negative Outcomes," Psychology and Developing Societies, , vol. 32(1), pages 94-121, March.
    13. Harrison, Mark, 2009. "Forging Success : Soviet Managers and False Accounting, 1943 to 1962," The Warwick Economics Research Paper Series (TWERPS) 909, University of Warwick, Department of Economics.
    14. Stephan B Bruns & John P A Ioannidis, 2016. "p-Curve and p-Hacking in Observational Research," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-13, February.
    15. Harrison, Mark, 2011. "Forging success: Soviet managers and accounting fraud, 1943-1962," Journal of Comparative Economics, Elsevier, vol. 39(1), pages 43-64, March.
    16. Necker, Sarah, 2014. "Scientific misbehavior in economics," Research Policy, Elsevier, vol. 43(10), pages 1747-1759.
    17. Gilles Grolleau & Naoufel Mzoughi, 2022. "How research institutions can make the best of scandals – once they become unavoidable," Post-Print hal-03908837, HAL.
    18. David Spiegelhalter, 2017. "Trust in numbers," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 948-965, October.
    19. Horbach, S.P.J.M.(Serge) & Halffman, W.(Willem), 2019. "The extent and causes of academic text recycling or ‘self-plagiarism’," Research Policy, Elsevier, vol. 48(2), pages 492-502.
    20. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:51:y:2022:i:2:p:499-526. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.