IDEAS home Printed from https://ideas.repec.org/p/mar/magkse/202504.html
   My bibliography  Save this paper

The Origins of Reporting Bias: Selective but Unbiased Reporting by Early-Career Researchers?

Author

Listed:
  • Anastasiya-Mariya Asanov

    (University of Kassel, INCHER and Institute of Economics)

  • Igor Asanov

    (University of Kassel, INCHER and Institute of Economics)

  • Guido Buenstorf

    (University of Kassel, INCHER and Institute of Economics)

  • Valon Kadriu

    (University of Kassel, INCHER and Institute of Economics)

  • Pia Schoch

    (University of Kassel, INCHER and Institute of Economics)

Abstract

Doctoral dissertations provide evidence about research practices in early-stage research. We examine reporting bias by manually collecting over 94,000 test statistics from a random sample of German dissertations and their follow-up papers worldwide. We observe selective reporting, as only a fraction of the tests in the dissertations is reported in follow-up papers. Unexpectedly, we find no increase in reporting bias in follow-up papers compared to dissertations nor, generally, reporting bias in dissertations or papers. Self-selection into higher-impact journals based on statistical significance may reconcile our finding of selective yet “unbiased†reporting with prior evidence suggesting pervasive reporting bias.

Suggested Citation

  • Anastasiya-Mariya Asanov & Igor Asanov & Guido Buenstorf & Valon Kadriu & Pia Schoch, 2025. "The Origins of Reporting Bias: Selective but Unbiased Reporting by Early-Career Researchers?," MAGKS Papers on Economics 202504, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    • Handle: RePEc:mar:magkse:202504
    as

    Download full text from publisher

    File URL: https://www.uni-marburg.de/en/fb02/research-groups/economics/macroeconomics/research/magks-joint-discussion-papers-in-economics/papers/2025-papers/04-2025.pdf
    File Function: First version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bruns, Stephan B. & Kalthaus, Martin, 2020. "Flexibility in the selection of patent counts: Implications for p-hacking and evidence-based policymaking," Research Policy, Elsevier, vol. 49(1).
    2. Abel Brodeur & Nikolai M. Cook & Jonathan S. Hartley & Anthony Heyes, 2024. "Do Preregistration and Preanalysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 527-561.
    3. Eva Vivalt, 2019. "Specification Searching and Significance Inflation Across Time, Methods and Disciplines," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 81(4), pages 797-816, August.
    4. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    5. Doucouliagos, Hristos & Hinz, Thomas & Zigova, Katarina, 2022. "Bias and careers: Evidence from the aid effectiveness literature," European Journal of Political Economy, Elsevier, vol. 71(C).
    6. Guido W. Imbens, 2021. "Statistical Significance, p-Values, and the Reporting of Uncertainty," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 157-174, Summer.
    7. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    8. Anastasiya-Mariya Asanov & Igor Asanov & Guido Buenstorf & Valon Kadriu & Pia Schoch, 2024. "Patterns of dissertation dissemination: publication-based outcomes of doctoral theses in the social sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(4), pages 2389-2405, April.
    9. Cilliers,Jacobus & Nour Elashmawy & David McKenzie, 2024. "Using Post-Double Selection Lasso in Field Experiments," Policy Research Working Paper Series 10931, The World Bank.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Asanov, Anastasiya-Mariya & Asanov, Igor & Buenstorf, Guido & Kadriu, Valon & Schoch, Pia, 2025. "The Origins of Reporting Bias: Selective but Unbiased Reporting by Early-Career Researchers?," I4R Discussion Paper Series 225, The Institute for Replication (I4R).
    2. Graham Elliott & Nikolay Kudrin & Kaspar Wuthrich, 2022. "The Power of Tests for Detecting $p$-Hacking," Papers 2205.07950, arXiv.org, revised Apr 2024.
    3. Costanza Naguib, 2024. "P-hacking and Significance Stars," Diskussionsschriften dp2409, Universitaet Bern, Departement Volkswirtschaft.
    4. Abel Brodeur, Nikolai M. Cook, Anthony Heyes, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," LCERPA Working Papers am0133, Laurier Centre for Economic Research and Policy Analysis.
    5. Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-Term Effects of the Targeting the Ultra-Poor Program - A Reproducibility and Replicability Assessment of Banerjee et al. (2021)," I4R Discussion Paper Series 142, The Institute for Replication (I4R).
    6. Graham Elliott & Nikolay Kudrin & Kaspar Wüthrich, 2022. "Detecting p‐Hacking," Econometrica, Econometric Society, vol. 90(2), pages 887-906, March.
    7. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments," GLO Discussion Paper Series 1157, Global Labor Organization (GLO).
    8. Abel Brodeur & Scott Carrell & David Figlio & Lester Lusher, 2023. "Unpacking P-hacking and Publication Bias," American Economic Review, American Economic Association, vol. 113(11), pages 2974-3002, November.
    9. Anna Dreber & Magnus Johannesson & Yifan Yang, 2024. "Selective reporting of placebo tests in top economics journals," Economic Inquiry, Western Economic Association International, vol. 62(3), pages 921-932, July.
    10. Jordan Adamson & Lucas Rentschler, 2023. "Criminal justice from a public choice perspective: an introduction to the special issue," Public Choice, Springer, vol. 196(3), pages 223-227, September.
    11. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    12. Brodeur, Abel & Cook, Nikolai & Hartley, Jonathan & Heyes, Anthony, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," MetaArXiv uxf39, Center for Open Science.
    13. Pettersson-Lidbom, Per, 2022. "Exit, Voice and Political Change: Evidence from Swedish Mass Migration to the United States. A Comment on Karadja and Prawitz (Journal of Political Economy, 2019)," Journal of Comments and Replications in Economics (JCRE), ZBW - Leibniz Information Centre for Economics, vol. 1, pages 1-13.
    14. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    15. Cantone, Giulio Giacomo, 2023. "The multiversal methodology as a remedy of the replication crisis," MetaArXiv kuhmz, Center for Open Science.
    16. Zohid Askarov & Anthony Doucouli & Hristos Doucouli & T D Stanley, 2023. "The Significance of Data-Sharing Policy," Journal of the European Economic Association, European Economic Association, vol. 21(3), pages 1191-1226.
    17. Roodman, David, 2025. "Opinion on the Replication Debate over Heyes and Saberian (2019)," I4R Discussion Paper Series 227, The Institute for Replication (I4R).
    18. repec:osf:metaar:kuhmz_v1 is not listed on IDEAS
    19. Aubry, Amandine & Héricourt, Jérôme & Marchal, Léa & Nedoncelle, Clément, 2022. "Does Immigration AffectWages? A Meta-Analysis," CEPREMAP Working Papers (Docweb) 2202, CEPREMAP.
    20. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    21. Alexandre Belloni & Victor Chernozhukov & Kengo Kato, 2019. "Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 749-758, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • A23 - General Economics and Teaching - - Economic Education and Teaching of Economics - - - Graduate
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mar:magkse:202504. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bernd Hayo (email available below). General contact details of provider: https://edirc.repec.org/data/vamarde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.