IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i1d10.1007_s11192-023-04872-6.html
   My bibliography  Save this article

Research assessment under debate: disentangling the interest around the DORA declaration on Twitter

Author

Listed:
  • Enrique Orduña-Malea

    (Universitat Politècnica de València)

  • Núria Bautista-Puig

    (Universidad Complutense de Madrid)

Abstract

Much debate has been around the misapplication of metrics in research assessment. As a result of this concern, the Declaration on Research Assessment (DORA) was launched, an initiative that caused opposing viewpoints. However, the discussion topics about DORA have not been formally identified, especially in participatory environments outside the scholarly communication process, such as social networks. This paper contributes to that end by analyzing 20,717 DORA-related tweets published from 2015 to 2022. The results show an increasing volume of tweets, mainly promotional and informative, but with limited participation of users, either commenting or engaging with the tweets, generating a scarcely polarized conversation driven primarily by a few DORA promoters. While a varied list of discussion topics is found (especially "Open science and research assessment," "Academics career assessment & innovation," and "Journal Impact Factor"), the DORA debate appears as part of broader conversations (research evaluation, open science). Further studies are needed to check whether these results are restricted to Twitter or reveal more general patterns. The findings might interest the different evaluators and evaluated agents regarding their interests and concerns around the reforms in the research evaluation.

Suggested Citation

  • Enrique Orduña-Malea & Núria Bautista-Puig, 2024. "Research assessment under debate: disentangling the interest around the DORA declaration on Twitter," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 537-559, January.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04872-6
    DOI: 10.1007/s11192-023-04872-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04872-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04872-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Di Costa, Flavia, 2019. "When research assessment exercises leave room for opportunistic behavior by the subjects under evaluation," Journal of Informetrics, Elsevier, vol. 13(3), pages 830-840.
    2. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    3. Deryl Northcott & Simon Linacre, 2010. "Producing Spaces for Academic Discourse: The Impact of Research Assessment Exercises and Journal Quality Rankings," Australian Accounting Review, CPA Australia, vol. 20(1), pages 38-54, March.
    4. repec:plo:pone00:0197265 is not listed on IDEAS
    5. Daniel Torres-Salinas & Wenceslao Arroyo-Machado & Nicolas Robinson-Garcia, 2023. "Bibliometric denialism," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5357-5359, September.
    6. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    7. Hendrik P. van Dalen & Kène Henkens, 2012. "Intended and unintended consequences of a publish‐or‐perish culture: A worldwide survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(7), pages 1282-1293, July.
    8. Kerrington Powell & Alyson Haslam & Vinay Prasad, 2022. "The Kardashian Index: a study of researchers' opinions on twitter 2014–2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1923-1930, April.
    9. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    10. Zhichao Fang & Rodrigo Costas & Paul Wouters, 2022. "User engagement with scholarly tweets of scientific papers: a large-scale and cross-disciplinary analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4523-4546, August.
    11. David Moher & Lex Bouter & Sabine Kleinert & Paul Glasziou & Mai Har Sham & Virginia Barbour & Anne-Marie Coriat & Nicole Foeger & Ulrich Dirnagl, 2020. "The Hong Kong Principles for assessing researchers: Fostering research integrity," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-14, July.
    12. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    13. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Correction to: Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 907-908, January.
    14. Mike Thelwall & Kayvan Kousha, 2021. "Researchers’ attitudes towards the h-index on Twitter 2007–2020: criticism and acceptance," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5361-5368, June.
    15. Hajar Sotudeh & Zeinab Saber & Farzin Ghanbari Aloni & Mahdieh Mirzabeigi & Farshad Khunjush, 2022. "A longitudinal study of the evolution of opinions about open access and its main features: a twitter sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5587-5611, October.
    16. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    17. Lorna Wildgaard, 2015. "A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 873-906, September.
    18. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 871-906, January.
    19. repec:plo:pbio00:2004089 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    2. Luciano Rossoni & Rodrigo Assunção Rosa, 2024. "Reducing the Matthew Effect on Journal Citations through an Inclusive Indexing Logic: The Brazilian Spell (Scientific Periodicals Electronic Library) Experience," Publications, MDPI, vol. 12(1), pages 1-24, February.
    3. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    4. Dušan Nikolić & Dragan Ivanović & Lidija Ivanović, 2024. "An open-source tool for merging data from multiple citation databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4573-4595, July.
    5. Tony Ross-Hellauer & Thomas Klebel & Petr Knoth & Nancy Pontika, 2024. "Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure," Science and Public Policy, Oxford University Press, vol. 51(3), pages 337-351.
    6. Toluwase Asubiaro & Sodiq Onaolapo & David Mills, 2024. "Regional disparities in Web of Science and Scopus journal coverage," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1469-1491, March.
    7. Huegel, Matthias, 2024. "University scientists’ multiple goals achievement: Social capital and its impact on research performance and research commercialization," Technovation, Elsevier, vol. 135(C).
    8. Torres-Salinas, Daniel & Orduña-Malea, Enrique & Delgado-Vázquez, Ángel & Gorraiz, Juan & Arroyo-Machado, Wenceslao, 2024. "Foundations of Narrative Bibliometrics," Journal of Informetrics, Elsevier, vol. 18(3).
    9. Joselyne Solórzano & Fernando Morante-Carballo & Néstor Montalván-Burbano & Josué Briones-Bitar & Paúl Carrión-Mero, 2022. "A Systematic Review of the Relationship between Geotechnics and Disasters," Sustainability, MDPI, vol. 14(19), pages 1-31, October.
    10. Adriana Ana Maria Davidescu & Margareta-Stela Florescu & Liviu Cosmin Mosora & Mihaela Hrisanta Mosora & Eduard Mihai Manta, 2022. "A Bibliometric Analysis of Research Publications of the Bucharest University of Economic Studies in Time of Pandemics: Implications for Teachers’ Professional Publishing Activity," IJERPH, MDPI, vol. 19(14), pages 1-36, July.
    11. Mike Thelwall, 2023. "Are successful co-authors more important than first authors for publishing academic journal articles?," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2211-2232, April.
    12. Irina Gerasimov & Binita KC & Armin Mehrabian & James Acker & Michael P. McGuire, 2024. "Comparison of datasets citation coverage in Google Scholar, Web of Science, Scopus, Crossref, and DataCite," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 3681-3704, July.
    13. Fernando Morante-Carballo & Néstor Montalván-Burbano & Maribel Aguilar-Aguilar & Paúl Carrión-Mero, 2022. "A Bibliometric Analysis of the Scientific Research on Artisanal and Small-Scale Mining," IJERPH, MDPI, vol. 19(13), pages 1-29, July.
    14. Weishu Liu & Rong Ni & Guangyuan Hu, 2024. "Web of Science Core Collection’s coverage expansion: the forgotten Arts & Humanities Citation Index?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(2), pages 933-955, February.
    15. Mike Thelwall, 2021. "Alternative medicines worth researching? Citation analyses of acupuncture, chiropractic, homeopathy, and osteopathy 1996–2017," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8731-8747, October.
    16. Gabriel Alves Vieira & Jacqueline Leta, 2024. "biblioverlap: an R package for document matching across bibliographic datasets," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4513-4527, July.
    17. Christopher Hansen & Holger Steinmetz & Jörn Block, 2022. "How to conduct a meta-analysis in eight steps: a practical guide," Management Review Quarterly, Springer, vol. 72(1), pages 1-19, February.
    18. Michael Gusenbauer, 2022. "Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2683-2745, May.
    19. Kayvan Kousha & Mike Thelwall, 2024. "Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(3), pages 215-244, March.
    20. Shir Aviv-Reuven & Ariel Rosenfeld, 2023. "A logical set theory approach to journal subject classification analysis: intra-system irregularities and inter-system discrepancies in Web of Science and Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 157-175, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04872-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.