IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i1d10.1007_s11192-023-04872-6.html
   My bibliography  Save this article

Research assessment under debate: disentangling the interest around the DORA declaration on Twitter

Author

Listed:
  • Enrique Orduña-Malea

    (Universitat Politècnica de València)

  • Núria Bautista-Puig

    (Universidad Complutense de Madrid)

Abstract

Much debate has been around the misapplication of metrics in research assessment. As a result of this concern, the Declaration on Research Assessment (DORA) was launched, an initiative that caused opposing viewpoints. However, the discussion topics about DORA have not been formally identified, especially in participatory environments outside the scholarly communication process, such as social networks. This paper contributes to that end by analyzing 20,717 DORA-related tweets published from 2015 to 2022. The results show an increasing volume of tweets, mainly promotional and informative, but with limited participation of users, either commenting or engaging with the tweets, generating a scarcely polarized conversation driven primarily by a few DORA promoters. While a varied list of discussion topics is found (especially "Open science and research assessment," "Academics career assessment & innovation," and "Journal Impact Factor"), the DORA debate appears as part of broader conversations (research evaluation, open science). Further studies are needed to check whether these results are restricted to Twitter or reveal more general patterns. The findings might interest the different evaluators and evaluated agents regarding their interests and concerns around the reforms in the research evaluation.

Suggested Citation

  • Enrique Orduña-Malea & Núria Bautista-Puig, 2024. "Research assessment under debate: disentangling the interest around the DORA declaration on Twitter," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 537-559, January.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04872-6
    DOI: 10.1007/s11192-023-04872-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04872-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04872-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Mike Thelwall & Kayvan Kousha, 2021. "Researchers’ attitudes towards the h-index on Twitter 2007–2020: criticism and acceptance," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5361-5368, June.
    2. Hendrik P. van Dalen & Kène Henkens, 2012. "Intended and unintended consequences of a publish-or-perish culture: A worldwide survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1282-1293, July.
    3. Hajar Sotudeh & Zeinab Saber & Farzin Ghanbari Aloni & Mahdieh Mirzabeigi & Farshad Khunjush, 2022. "A longitudinal study of the evolution of opinions about open access and its main features: a twitter sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5587-5611, October.
    4. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    5. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Di Costa, Flavia, 2019. "When research assessment exercises leave room for opportunistic behavior by the subjects under evaluation," Journal of Informetrics, Elsevier, vol. 13(3), pages 830-840.
    6. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    7. Deryl Northcott & Simon Linacre, 2010. "Producing Spaces for Academic Discourse: The Impact of Research Assessment Exercises and Journal Quality Rankings," Australian Accounting Review, CPA Australia, vol. 20(1), pages 38-54, March.
    8. Zhichao Fang & Rodrigo Costas & Paul Wouters, 2022. "User engagement with scholarly tweets of scientific papers: a large-scale and cross-disciplinary analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4523-4546, August.
    9. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    10. repec:plo:pone00:0197265 is not listed on IDEAS
    11. Daniel Torres-Salinas & Wenceslao Arroyo-Machado & Nicolas Robinson-Garcia, 2023. "Bibliometric denialism," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5357-5359, September.
    12. David Moher & Lex Bouter & Sabine Kleinert & Paul Glasziou & Mai Har Sham & Virginia Barbour & Anne-Marie Coriat & Nicole Foeger & Ulrich Dirnagl, 2020. "The Hong Kong Principles for assessing researchers: Fostering research integrity," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-14, July.
    13. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    14. Lorna Wildgaard, 2015. "A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 873-906, September.
    15. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 871-906, January.
    16. Kerrington Powell & Alyson Haslam & Vinay Prasad, 2022. "The Kardashian Index: a study of researchers' opinions on twitter 2014–2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1923-1930, April.
    17. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    18. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Correction to: Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 907-908, January.
    19. repec:plo:pbio00:2004089 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    2. Luciano Rossoni & Rodrigo Assunção Rosa, 2024. "Reducing the Matthew Effect on Journal Citations through an Inclusive Indexing Logic: The Brazilian Spell (Scientific Periodicals Electronic Library) Experience," Publications, MDPI, vol. 12(1), pages 1-24, February.
    3. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    4. Mike Thelwall, 2021. "Alternative medicines worth researching? Citation analyses of acupuncture, chiropractic, homeopathy, and osteopathy 1996–2017," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8731-8747, October.
    5. Dušan Nikolić & Dragan Ivanović & Lidija Ivanović, 2024. "An open-source tool for merging data from multiple citation databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4573-4595, July.
    6. Gabriel Alves Vieira & Jacqueline Leta, 2024. "biblioverlap: an R package for document matching across bibliographic datasets," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4513-4527, July.
    7. Christopher Hansen & Holger Steinmetz & Jörn Block, 2022. "How to conduct a meta-analysis in eight steps: a practical guide," Management Review Quarterly, Springer, vol. 72(1), pages 1-19, February.
    8. Tony Ross-Hellauer & Thomas Klebel & Petr Knoth & Nancy Pontika, 2024. "Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure," Science and Public Policy, Oxford University Press, vol. 51(3), pages 337-351.
    9. Michael Gusenbauer, 2022. "Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2683-2745, May.
    10. Toluwase Asubiaro & Sodiq Onaolapo & David Mills, 2024. "Regional disparities in Web of Science and Scopus journal coverage," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1469-1491, March.
    11. Kayvan Kousha & Mike Thelwall, 2024. "Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(3), pages 215-244, March.
    12. Nikolaos Efthimiou & Thomas Giotis & Athanasios Ragkos, 2025. "Applications for Non-Conventional Water Resources in the Mediterranean Basin: A Literature Review," Sustainability, MDPI, vol. 17(11), pages 1-30, May.
    13. Shir Aviv-Reuven & Ariel Rosenfeld, 2023. "A logical set theory approach to journal subject classification analysis: intra-system irregularities and inter-system discrepancies in Web of Science and Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 157-175, January.
    14. Pingluo Xue & Chongyang Shen & Huaizhi Tang & Yunjia Liu & Yuanfang Huang, 2024. "Knowledge Atlas of Cultivated Land Quality Evaluation Based on Web of Science Since the 21st Century (2000–2023)," Land, MDPI, vol. 13(10), pages 1-15, October.
    15. Abdelkebir Sahid & Yassine Maleh & Karim Ouazzane, 2025. "Changing landscape of fake news research on social media: a bibliometric analysis," Quality & Quantity: International Journal of Methodology, Springer, vol. 59(2), pages 901-954, April.
    16. Huegel, Matthias, 2024. "University scientists’ multiple goals achievement: Social capital and its impact on research performance and research commercialization," Technovation, Elsevier, vol. 135(C).
    17. Adriane Cavalieri & João Reis & Marlene Amorim, 2021. "Circular Economy and Internet of Things: Mapping Science of Case Studies in Manufacturing Industry," Sustainability, MDPI, vol. 13(6), pages 1-32, March.
    18. Torres-Salinas, Daniel & Orduña-Malea, Enrique & Delgado-Vázquez, Ángel & Gorraiz, Juan & Arroyo-Machado, Wenceslao, 2024. "Foundations of Narrative Bibliometrics," Journal of Informetrics, Elsevier, vol. 18(3).
    19. Delgado-Quirós, Lorena & Ortega, José Luis, 2025. "Citation counts and inclusion of references in seven free-access scholarly databases: A comparative analysis," Journal of Informetrics, Elsevier, vol. 19(1).
    20. Joselyne Solórzano & Fernando Morante-Carballo & Néstor Montalván-Burbano & Josué Briones-Bitar & Paúl Carrión-Mero, 2022. "A Systematic Review of the Relationship between Geotechnics and Disasters," Sustainability, MDPI, vol. 14(19), pages 1-31, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04872-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.