IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v50y2016i2p901-917.html

Publication bias in the German social sciences: an application of the caliper test to three top-tier German social science journals

Author

Listed:
  • Carl Berning

  • Bernd Weiß

Abstract

Systematic research reviews have become essential in all empirical sciences. However, the validity of research syntheses is threatened if the preparation, submission or publication of research findings depends on the statistical significance of these findings. The present study investigates publication bias in three top-tier journals in the German social sciences, utilizing the caliper test. For the period between 2001 and 2010, we have collected 156 articles that appeared in the Kölner Zeitschrift für Soziologie und Sozialpsychologie (KZfSS), the Zeitschrift für Soziologie (ZfS) and the Politische Vierteljahresschrift (PVS). In all three journals, we found empirical evidence for the existence of publication bias at the 10 % significance level. We also investigated possible causes linked to this bias, including single versus multiple authorship as well as academic degree. We found only weak support for the relationships between individual author characteristics and publication bias. Copyright Springer Science+Business Media Dordrecht 2016

Suggested Citation

  • Carl Berning & Bernd Weiß, 2016. "Publication bias in the German social sciences: an application of the caliper test to three top-tier German social science journals," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(2), pages 901-917, March.
    • Handle: RePEc:spr:qualqt:v:50:y:2016:i:2:p:901-917
    DOI: 10.1007/s11135-015-0182-4
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11135-015-0182-4
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11135-015-0182-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Kerry Dwan & Douglas G Altman & Juan A Arnaiz & Jill Bloom & An-Wen Chan & Eugenia Cronin & Evelyne Decullier & Philippa J Easterbrook & Erik Von Elm & Carrol Gamble & Davina Ghersi & John P A Ioannid, 2008. "Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias," PLOS ONE, Public Library of Science, vol. 3(8), pages 1-31, August.
    2. Weiß Bernd & Wagner Michael, 2011. "The Identification and Prevention of Publication Bias in the Social Sciences and Economics," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(5-6), pages 661-684, October.
    3. Peter A. Lawrence, 2003. "The politics of publication," Nature, Nature, vol. 422(6929), pages 259-261, March.
    4. Peder Olesen Larsen & Markus Ins, 2010. "The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 575-603, September.
    5. Auspurg Katrin & Hinz Thomas, 2011. "What Fuels Publication Bias?: Theoretical and Empirical Analyses of Risk Factors Using the Caliper Test," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(5-6), pages 636-660, October.
    6. Aickin, M. & Gensler, H., 1996. "Adjusting for multiple testing when reporting research results: The Bonferroni vs Holm methods," American Journal of Public Health, American Public Health Association, vol. 86(5), pages 726-728.
    7. Michael Graber & Andrey Launov & Klaus Wälde, 2008. "Publish or Perish? The Increasing Importance of Publications for Prospective Economics Professors in Austria, Germany and Switzerland," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 457-472, November.
    8. repec:bla:germec:v:9:y:2008:i::p:457-472 is not listed on IDEAS
    9. De Long, J Bradford & Lang, Kevin, 1992. "Are All Economic Hypotheses False?," Journal of Political Economy, University of Chicago Press, vol. 100(6), pages 1257-1272, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bruns, Stephan B. & Asanov, Igor & Bode, Rasmus & Dunger, Melanie & Funk, Christoph & Hassan, Sherif M. & Hauschildt, Julia & Heinisch, Dominik & Kempa, Karol & König, Johannes & Lips, Johannes & Verb, 2019. "Reporting errors and biases in published empirical findings: Evidence from innovation research," Research Policy, Elsevier, vol. 48(9), pages 1-1.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    2. Paolo Crosetto & Antonio Filippin & Janna Heider, 2015. "A Study of Outcome Reporting Bias Using Gender Differences in Risk Attitudes," CESifo Economic Studies, CESifo Group, vol. 61(1), pages 239-262.
    3. Auspurg Katrin & Hinz Thomas, 2011. "What Fuels Publication Bias?: Theoretical and Empirical Analyses of Risk Factors Using the Caliper Test," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(5-6), pages 636-660, October.
    4. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    5. Marco Pautasso, 2012. "Publication Growth in Biological Sub-Fields: Patterns, Predictability and Sustainability," Sustainability, MDPI, vol. 4(12), pages 1-14, November.
    6. Marco Pautasso & Hanno Schäfer, 2010. "Peer review delay and selectivity in ecology journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 307-315, August.
    7. Mark D Lindner & Richard K Nakamura, 2015. "Examining the Predictive Validity of NIH Peer Review Scores," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-12, June.
    8. Darren Grant & William Green, 2013. "Grades as incentives," Empirical Economics, Springer, vol. 44(3), pages 1563-1592, June.
    9. Dominika Reckova & Zuzana Irsova, 2015. "Publication Bias in Measuring Anthropogenic Climate Change," Energy & Environment, , vol. 26(5), pages 853-862, September.
    10. Klaus Wohlrabe, 2018. "Einige Anmerkungen zum FAZ-Ökonomenranking 2018," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 71(20), pages 29-33, October.
    11. Alexander Libman & Joachim Zweynert, 2014. "Ceremonial Science: The State of Russian Economics Seen Through the Lens of the Work of ‘Doctor of Science’ Candidates," Working Papers 337, Leibniz Institut für Ost- und Südosteuropaforschung (Leibniz Institute for East and Southeast European Studies).
    12. Rajeev Dehejia, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series wp-2013-011, World Institute for Development Economic Research (UNU-WIDER).
    13. Hyojung Tak & Gregory Ruhnke & Ya-Chen Shih, 2015. "The Association between Patient-Centered Attributes of Care and Patient Satisfaction," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 8(2), pages 187-197, April.
    14. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    15. Stephen A Gallo & Afton S Carpenter & David Irwin & Caitlin D McPartland & Joseph Travis & Sofie Reynders & Lisa A Thompson & Scott R Glisson, 2014. "The Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies," PLOS ONE, Public Library of Science, vol. 9(9), pages 1-9, September.
    16. Alexander Butz & Klaus Wohlrabe, 2016. "Die Ökonomen-Rankings 2015 von Handelsblatt, FAZ und RePEc: Methodik, Ergebnisse, Kritik und Vergleich," ifo Working Paper Series 212, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    17. Havranek, Tomas & Irsova, Zuzana, 2011. "Estimating vertical spillovers from FDI: Why results vary and what the true effect is," Journal of International Economics, Elsevier, vol. 85(2), pages 234-244.
    18. Rolf Sternberg, 2015. "The publication and citation behaviour of economic geographers and geographical economists compared," Review of Regional Research: Jahrbuch für Regionalwissenschaft, Springer;Gesellschaft für Regionalforschung (GfR), vol. 35(1), pages 1-27, February.
    19. Jeffrey C. Valentine & Sandra Jo Wilson & David Rindskopf & Timothy S. Lau & Emily E. Tanner-Smith & Martha Yeide & Robin LaSota & Lisa Foster, 2017. "Synthesizing Evidence in Public Policy Contexts," Evaluation Review, , vol. 41(1), pages 3-26, February.
    20. Alexander Kalgin & Olga Kalgina & Anna Lebedeva, 2019. "Publication Metrics as a Tool for Measuring Research Productivity and Their Relation to Motivation," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 44-86.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:50:y:2016:i:2:p:901-917. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.