IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v8y2018i1p2158244018758836.html
   My bibliography  Save this article

Web of Science: Showing a Bug Today That Can Mislead Scientific Research Output Prediction

Author

Listed:
  • Pablo Diniz Batista
  • Igor Marques-Carneiro
  • Leduc Hermeto de Almeida Fauth
  • Márcia de Oliveira Reis Brandão

Abstract

As in all domains of human activity, economic issues and the increase of people working in scientific research have altered the way scientific production is evaluated as well as the objectives for performing the evaluation. The h index was introduced in 2005 by J. E. Hirsch as an indicator for the measurement of individual scientific output not only in terms of quantity but also in terms of quality, and its use has since spread throughout the world. In 2007, Hirsch proposed its adoption as the best way to predict future scientific achievement and, consequently, as a useful guide for investment in research and for institutions when hiring members for their scientific staff. Since then, several authors have also been using the Thomson ISI Web of Science database to develop their proposals for evaluating research output. Here, using a software we have developed, we analyze more than 100,000 articles and show that a subtle flaw in Web of Science can inflate the results of information collected, therefore compromising the exactness and, consequently, the effectiveness of Hirsch’s proposal and its variations.

Suggested Citation

  • Pablo Diniz Batista & Igor Marques-Carneiro & Leduc Hermeto de Almeida Fauth & Márcia de Oliveira Reis Brandão, 2018. "Web of Science: Showing a Bug Today That Can Mislead Scientific Research Output Prediction," SAGE Open, , vol. 8(1), pages 21582440187, February.
    • Handle: RePEc:sae:sagope:v:8:y:2018:i:1:p:2158244018758836
    DOI: 10.1177/2158244018758836
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244018758836
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2158244018758836?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Pablo D. Batista & Mônica G. Campiteli & Osame Kinouchi, 2006. "Is it possible to compare researchers with different scientific interests?," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(1), pages 179-189, July.
    2. Daniel E. Acuna & Stefano Allesina & Konrad P. Kording, 2012. "Predicting scientific success," Nature, Nature, vol. 489(7415), pages 201-202, September.
    3. Amin Mazloumian, 2012. "Predicting Scholars' Scientific Impact," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-5, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. de Carvalho, Gustavo Dambiski Gomes & Sokulski, Carla Cristiane & da Silva, Wesley Vieira & de Carvalho, Hélio Gomes & de Moura, Rafael Vignoli & de Francisco, Antonio Carlos & da Veiga, Claudimar Per, 2020. "Bibliometrics and systematic reviews: A comparison between the Proknow-C and the Methodi Ordinatio," Journal of Informetrics, Elsevier, vol. 14(3).
    2. James Hartley, 2019. "Some reflections on being cited 10,000 times," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 375-381, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lisa Geraci & Steve Balsis & Alexander J. Busch Busch, 2015. "Gender and the h index in psychology," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2023-2034, December.
    2. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    3. Yuhao Zhou & Ruijie Wang & An Zeng, 2022. "Predicting the impact and publication date of individual scientists’ future papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1867-1882, April.
    4. João A G Moreira & Xiao Han T Zeng & Luís A Nunes Amaral, 2015. "The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-17, November.
    5. Matthias Kuppler, 2022. "Predicting the future impact of Computer Science researchers: Is there a gender bias?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6695-6732, November.
    6. Akella, Akhil Pandey & Alhoori, Hamed & Kondamudi, Pavan Ravikanth & Freeman, Cole & Zhou, Haiming, 2021. "Early indicators of scientific impact: Predicting citations with altmetrics," Journal of Informetrics, Elsevier, vol. 15(2).
    7. Li Hou & Qiang Wu & Yundong Xie, 2022. "Does early publishing in top journals really predict long-term scientific success in the business field?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6083-6107, November.
    8. Zhiya Zuo & Kang Zhao, 2021. "Understanding and predicting future research impact at different career stages—A social network perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(4), pages 454-472, April.
    9. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    10. Kaur, Jasleen & Radicchi, Filippo & Menczer, Filippo, 2013. "Universality of scholarly impact metrics," Journal of Informetrics, Elsevier, vol. 7(4), pages 924-932.
    11. Hyeonchae Yang & Woo-Sung Jung, 2015. "A strategic management approach for Korean public research institutes based on bibliometric investigation," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(4), pages 1437-1464, July.
    12. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    13. Marco Giuliani & Stefano Marasca, 2015. "La valutazione della ricerca tramite indici bibliometrici: riflessioni da una prospettiva economico-aziendale," MANAGEMENT CONTROL, FrancoAngeli Editore, vol. 2015(1), pages 133-151.
    14. Letchford, Adrian & Preis, Tobias & Moat, Helen Susannah, 2016. "The advantage of simple paper abstracts," Journal of Informetrics, Elsevier, vol. 10(1), pages 1-8.
    15. Zoltán Krajcsák, 2021. "Researcher Performance in Scopus Articles ( RPSA ) as a New Scientometric Model of Scientific Output: Tested in Business Area of V4 Countries," Publications, MDPI, vol. 9(4), pages 1-23, October.
    16. Franceschini, Fiorenzo & Maisano, Domenico, 2011. "Structured evaluation of the scientific output of academic research groups by recent h-based indicators," Journal of Informetrics, Elsevier, vol. 5(1), pages 64-74.
    17. Lin Zhang & Wolfgang Glänzel, 2012. "Where demographics meets scientometrics: towards a dynamic career analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 617-630, May.
    18. Shahryar Rahnamayan & Sedigheh Mahdavi & Kalyanmoy Deb & Azam Asilian Bidgoli, 2020. "Ranking Multi-Metric Scientific Achievements Using a Concept of Pareto Optimality," Mathematics, MDPI, vol. 8(6), pages 1-46, June.
    19. Hyungjo Hur & Maryam A Andalib & Julie A Maurer & Joshua D Hawley & Navid Ghaffarzadegan, 2017. "Recent trends in the U.S. Behavioral and Social Sciences Research (BSSR) workforce," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-18, February.
    20. Serge Galam, 2011. "Tailor based allocations for multiple authorship: a fractional gh-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 365-379, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:8:y:2018:i:1:p:2158244018758836. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.