IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp11041.html
   My bibliography  Save this paper

A Walk on the Wild Side: 'Predatory' Journals and Information Asymmetries in Scientific Evaluations

Author

Listed:
  • Bagues, Manuel

    (University of Warwick)

  • Sylos-Labini, Mauro

    (University of Pisa)

  • Zinovyeva, Natalia

    (University of Warwick)

Abstract

In recent years the academic world has experienced a mushrooming of journals that falsely pretend to be legitimate academic outlets. We study this phenomenon using information from 46,000 researchers seeking promotion in Italian academia. About 5% of them have published in journals included in the blacklist of 'potential, possible, or probable predatory journals' elaborated by the scholarly librarian Jeffrey Beall. Data from a survey that we conducted among these researchers confirms that at least one third of these journals do not provide peer review or they engage in some other type of irregular editorial practice. We identify two factors that may have spurred publications in dubious journals. First, some of these journals have managed to be included in citation indexes such as Scopus that many institutions consider as a guarantee of quality. Second, we show that authors who publish in these journals are more likely to receive a positive evaluation when (randomly selected) scientific evaluators lack research expertise. Overall, our analysis suggests that the proliferation of 'predatory' journals may reflect the existence of severe information asymmetries in scientific evaluations.

Suggested Citation

  • Bagues, Manuel & Sylos-Labini, Mauro & Zinovyeva, Natalia, 2017. "A Walk on the Wild Side: 'Predatory' Journals and Information Asymmetries in Scientific Evaluations," IZA Discussion Papers 11041, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp11041
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp11041.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    2. Manuel Bagues & Mauro Sylos-Labini & Natalia Zinovyeva, 2017. "Does the Gender Composition of Scientific Committees Matter?," American Economic Review, American Economic Association, vol. 107(4), pages 1207-1238, April.
    3. Jeffrey Beall, 2012. "Predatory publishers are corrupting open access," Nature, Nature, vol. 489(7415), pages 179-179, September.
    4. Robert Gibbons, 1998. "Incentives in Organizations," Journal of Economic Perspectives, American Economic Association, vol. 12(4), pages 115-132, Fall.
    5. Natalia Zinovyeva & Manuel Bagues, 2015. "The Role of Connections in Academic Promotions," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 264-292, April.
    6. Bagues, Manuel & Sylos-Labini, Mauro & Zinovyeva, Natalia, 2019. "Connections in scientific committees and applicants’ self-selection: Evidence from a natural randomized experiment," Labour Economics, Elsevier, vol. 58(C), pages 81-97.
    7. Pierre Régibeau & Katharine E. Rockett, 2016. "Research assessment and recognized excellence: simple bibliometrics for more efficient academic research evaluations," Economic Policy, CEPR;CES;MSH, vol. 31(88), pages 611-652.
    8. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    9. Holmstrom, Bengt & Milgrom, Paul, 1991. "Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 7(0), pages 24-52, Special I.
    10. Jingfeng Xia & Jennifer L. Harmon & Kevin G. Connolly & Ryan M. Donnelly & Mary R. Anderson & Heather A. Howard, 2015. "Who publishes in “predatory” journals?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1406-1417, July.
    11. Rebora, Gianfranco & Turri, Matteo, 2013. "The UK and Italian research assessment exercises face to face," Research Policy, Elsevier, vol. 42(9), pages 1657-1666.
    12. Imad A. Moosa, 2016. "A Critique of the Bucket Classification of Journals: The ABDC List as an Example," The Economic Record, The Economic Society of Australia, vol. 92(298), pages 448-463, September.
    13. David A. King, 2004. "The scientific impact of nations," Nature, Nature, vol. 430(6997), pages 311-316, July.
    14. Alex Csiszar, 2016. "Peer review: Troubled from the start," Nature, Nature, vol. 532(7599), pages 306-308, April.
    15. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    16. Piotr Sorokowski & Emanuel Kulczycki & Agnieszka Sorokowska & Katarzyna Pisanski, 2017. "Predatory journals recruit fake editor," Nature, Nature, vol. 543(7646), pages 481-483, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tatiana Marina & Ivan Sterligov, 2021. "Prevalence of potentially predatory publishing in Scopus on the country level," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5019-5077, June.
    2. Briony Swire-Thompson & David Lazer, 2022. "Reducing Health Misinformation in Science: A Call to Arms," The ANNALS of the American Academy of Political and Social Science, , vol. 700(1), pages 124-135, March.
    3. Balatskiy, E. & Yurevich, M., 2021. "Russian economic science on the international market of "predatory" publications," Journal of the New Economic Association, New Economic Association, vol. 50(2), pages 190-198.
    4. Mohamed Boufarss & Mikael Laakso, 2020. "Open Sesame? Open access priorities, incentives, and policies among higher education institutions in the United Arab Emirates," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1553-1577, August.
    5. Mark Armstrong, 2021. "Plan S: An economist's perspective," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(8), pages 2017-2026, December.
    6. Andrea Cortegiani & Andrea Manca & Manoj Lalu & David Moher, 2020. "Inclusion of predatory journals in Scopus is inflating scholars’ metrics and advancing careers," International Journal of Public Health, Springer;Swiss School of Public Health (SSPH+), vol. 65(1), pages 3-4, January.
    7. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    8. Vít Macháček & Martin Srholec, 2021. "RETRACTED ARTICLE: Predatory publishing in Scopus: evidence on cross-country differences," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 1897-1921, March.
    9. Andrei V. Grinëv & Daria S. Bylieva & Victoria V. Lobatyuk, 2021. "Russian University Teachers’ Perceptions of Scientometrics," Publications, MDPI, vol. 9(2), pages 1-16, May.
    10. Bo-Christer Björk & Sari Kanto-Karvonen & J. Tuomas Harviainen, 2020. "How Frequently Are Articles in Predatory Open Access Journals Cited," Publications, MDPI, vol. 8(2), pages 1-12, March.
    11. You, Taekho & Park, Jinseo & Lee, June Young & Yun, Jinhyuk & Jung, Woo-Sung, 2022. "Disturbance of questionable publishing to academia," Journal of Informetrics, Elsevier, vol. 16(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohamed Boufarss & Mikael Laakso, 2020. "Open Sesame? Open access priorities, incentives, and policies among higher education institutions in the United Arab Emirates," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1553-1577, August.
    2. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    3. Daniele Checchi & Silvia Poli & Enrico Rettore, 2018. "Does Random Selection of Selectors Improve the Quality of Selected Candidates? An Investigation in the Italian Academia," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 4(2), pages 211-247, July.
    4. Demir, Selcuk Besir, 2018. "Predatory journals: Who publishes in them and why?," Journal of Informetrics, Elsevier, vol. 12(4), pages 1296-1311.
    5. Margit Osterloh & Bruno S. Frey, 2009. "Research Governance in Academia: Are there Alternatives to Academic Rankings?," CREMA Working Paper Series 2009-17, Center for Research in Economics, Management and the Arts (CREMA).
    6. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    7. Battistin, Erich & Ovidi, Marco, 2017. "Rising Stars," IZA Discussion Papers 11198, Institute of Labor Economics (IZA).
    8. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    9. Richard McManus & Karen Mumford & Cristina Sechel, 2022. "Measuring research excellence amongst economics lecturers in the UK," Bulletin of Economic Research, Wiley Blackwell, vol. 74(2), pages 386-404, April.
    10. Jappelli, Tullio & Nappi, Carmela Anna & Torrini, Roberto, 2017. "Gender effects in research evaluation," Research Policy, Elsevier, vol. 46(5), pages 911-924.
    11. Thiele, Veikko, 2007. "Performance measurement in multi-task agencies," Research in Economics, Elsevier, vol. 61(3), pages 148-163, September.
    12. Maria Cotofan, 2019. "Learning from Praise: Evidence from a Field Experiment with Teachers," Tinbergen Institute Discussion Papers 19-082/V, Tinbergen Institute.
    13. Luis Garicano & Richard A. Posner, 2005. "Intelligence Failures: An Organizational Economics Perspective," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 151-170, Fall.
    14. Krasteva, Silvana & Sharma, Priyanka & Wagman, Liad, 2015. "The 80/20 rule: Corporate support for innovation by employees," International Journal of Industrial Organization, Elsevier, vol. 38(C), pages 32-43.
    15. Erich Battistin & Marco Ovidi, 2022. "Rising Stars: Expert Reviews and Reputational Yardsticks in the Research Excellence Framework," Economica, London School of Economics and Political Science, vol. 89(356), pages 830-848, October.
    16. Sarena F. Goodman & Lesley J. Turner, 2013. "The Design of Teacher Incentive Pay and Educational Outcomes: Evidence from the New York City Bonus Program," Journal of Labor Economics, University of Chicago Press, vol. 31(2), pages 409-420.
    17. Carmen Osuna & Laura Cruz Castro & Luis Sanz Menéndez, 2010. "Knocking down some Assumptions about the Effects of Evaluation Systems on Publications," Working Papers 1010, Instituto de Políticas y Bienes Públicos (IPP), CSIC.
    18. Zacchia, Giulia, 2016. "Segregation or homologation? Gender differences in recent Italian economic thought," MPRA Paper 72279, University Library of Munich, Germany.
    19. Pierre Deschamps, 2018. "Gender Quotas in Hiring Committees: a Boon or a Bane for Women?," Sciences Po publications 82, Sciences Po.
    20. Jirjahn, Uwe & Mohrenweiser, Jens, 2023. "Variable Payment Schemes and Productivity: Do Individual-Based Schemes Really Have a Stronger Influence than Collective Ones?," IZA Discussion Papers 16267, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    scientific misconduct; academic evaluations;

    JEL classification:

    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp11041. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.