IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v130y2025i3d10.1007_s11192-025-05254-w.html
   My bibliography  Save this article

How to use assignments of United Nations sustainable development goals (SDGs) to scientific papers in research evaluation? The proposal of a gold standard combining assignments from different data providers

Author

Listed:
  • Rüdiger Mutz

    (University of Zurich)

  • Lutz Bornmann

    (Administrative Headquarters of the Max Planck Society
    Max Planck Institute for Solid State Research)

  • Robin Haunschild

    (Max Planck Institute for Solid State Research)

Abstract

To identify research that addresses the biggest problems facing the world today, researchers have used the United Nations (UN) Sustainable Development Goals (SDGs) as a blueprint for measuring the societal impact of research. Although one might assume that the mapping of papers to the SDGs has a high degree of uniformity (based more or less on the same data), empirical results show that this is not the case: different mapping approaches lead to different assignments. The central aim of this paper is to use the National Open Research Analytics (NORA, Technical University of Denmark) database not only to test the agreement between four SDG mapping approaches [Web of Science (WoS)TM, OSDG, Scopus-ML, Scopus-SM], but also to investigate whether an overarching SDG mapping can be found (a gold standard). N = 526,520 Danish publications are analyzed with respect to the following aspects: distribution of SDGs over publications, agreement between SDG mapping approaches, prediction of SDG assignments, and the gold standard combining assignments from different mapping approaches (occupancy analysis). The main findings are as follows: 35.5% of the documents have at least one SDG assignment. The SDG occupancy is low, with the highest occupancy and detection probability for SDG 3 “Good health and well-being”. Agreement between SDG mapping approaches is low to moderate. The result of the gold standard analysis is not only the determination of a “true” SDG status of a publication and the frequency of the SDG assignments (occupancy), but also the uncertainty associated with this SDG assignment (detection probability).

Suggested Citation

  • Rüdiger Mutz & Lutz Bornmann & Robin Haunschild, 2025. "How to use assignments of United Nations sustainable development goals (SDGs) to scientific papers in research evaluation? The proposal of a gold standard combining assignments from different data pro," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1519-1546, March.
    • Handle: RePEc:spr:scient:v:130:y:2025:i:3:d:10.1007_s11192-025-05254-w
    DOI: 10.1007/s11192-025-05254-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-025-05254-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-025-05254-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sachs,Jeffrey D. & Kroll,Christian & Lafortune,Guillame & Fuller,Grayson & Woelm,Finn, 2022. "Sustainable Development Report 2022," Cambridge Books, Cambridge University Press, number 9781009210034, Enero-Abr.
    2. Arash Hajikhani & Arho Suominen, 2022. "Mapping the sustainable development goals (SDGs) in science, technology and innovation: application of machine learning in SDG-oriented artefact detection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6661-6693, November.
    3. Sachs,Jeffrey D. & Kroll,Christian & Lafortune,Guillame & Fuller,Grayson & Woelm,Finn, 2022. "Sustainable Development Report 2022," Cambridge Books, Cambridge University Press, number 9781009210089, Enero-Abr.
    4. Chinyereugo M Umemneku Chikere & Kevin Wilson & Sara Graziadio & Luke Vale & A Joy Allen, 2019. "Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard – An update," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-25, October.
    5. Michael Eid, 2000. "A multitrait-multimethod model with minimal assumptions," Psychometrika, Springer;The Psychometric Society, vol. 65(2), pages 241-261, June.
    6. Shaher H. Zyoud, 2023. "Analyzing and visualizing global research trends on COVID-19 linked to sustainable development goals," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 25(6), pages 5459-5493, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Junuguru Srinivas & Naveen Kolloju & Akanksha Singh & Siriman Naveen & Sudhaveni Naresh, 2024. "The COVID-19 Pandemic and Its Impact on Sustainable Development Goals-2030," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(4), pages 15485-15498, December.
    2. Fareri, Silvia & Apreda, Riccardo & Mulas, Valentina & Alonso, Ruben, 2023. "The worker profiler: Assessing the digital skill gaps for enhancing energy efficiency in manufacturing," Technological Forecasting and Social Change, Elsevier, vol. 196(C).
    3. Kyoko Sasaki & Wendy Stubbs & Megan Farrelly, 2023. "The relationship between corporate purpose and the sustainable development goals in large Japanese companies," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 30(5), pages 2475-2489, September.
    4. TRIARCHI Eirini & KOLIAS Georgios & KYPRIOTELIS Efstratios, 2023. "Exploring the Role of Institutional Governance in Climate Action in the Western Balkans Economies," European Journal of Interdisciplinary Studies, Bucharest Economic Academy, issue 02, June.
    5. Gamze Sart, 2022. "Impact of Higher Education and Globalization on Sustainable Development in the New EU Member States," Sustainability, MDPI, vol. 14(19), pages 1-13, September.
    6. Christoph Funk & Elena Tönjes & Ramona Teuber & Lutz Breuer, 2024. "Reading between the lines: The intersection of research attention and sustainable development goals," Sustainable Development, John Wiley & Sons, Ltd., vol. 32(5), pages 4545-4566, October.
    7. de Moura, Fernanda Senra & Barbrook-Johnson, Peter, 2022. "Using data-driven systems mapping to contextualise complexity economics insights," INET Oxford Working Papers 2022-27, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford.
    8. Rakotoarimanana Zy Misa Harivelo & Rakotoarimanana Zy Harifidy, 2022. "A Review of Environmental Protection and Sustainable Development in Madagascar," J, MDPI, vol. 5(4), pages 1-20, November.
    9. Audronė Urmanavičienė & Eglė Butkevičienė, 2023. "The Role of Work Integration Social Enterprises in Achieving Sustainable Development Goals during the COVID-19 Pandemic: The Case Study of Lithuania," Sustainability, MDPI, vol. 15(5), pages 1-21, February.
    10. Stefana Matović & Dunja Demirović Bajrami & Tamara Gajić & Milica Lakić & Darko Martinov & Srđan Šuput & Adriana Radosavac & Ružica Đervida, 2024. "Beyond Numbers: Challenges in Measuring SDG4 Targets—Serbia’s Perspective," Sustainability, MDPI, vol. 16(22), pages 1-24, November.
    11. Delpy, Léo, 2024. "Protection or pressure? reciprocity in informal social protection in southern Madagascar," World Development Perspectives, Elsevier, vol. 34(C).
    12. Abbas, Shujaat & Sinha, Avik & Saha, Tanaya & Shah, Muhammad Ibrahim, 2023. "Response of mineral market to renewable energy production in the USA: Where lies the sustainable energy future," Energy Policy, Elsevier, vol. 182(C).
    13. Mohammad Mizenur Rahaman & Shamima Akter & Md Alamgir Hossain & Adiba Rahman Bushra Chowdhury & Renhong Wu, 2024. "Green accounting and reporting in Bangladesh’s pharmaceutical and textile industries: A holistic perspective," PLOS ONE, Public Library of Science, vol. 19(9), pages 1-22, September.
    14. World Bank Group, 2023. "Cambodia Country Climate and Development Report," World Bank Publications - Reports 40467, The World Bank Group.
    15. Torrisi, Orsola, 2024. "Violent instability and modern contraception: Evidence from Mali," World Development, Elsevier, vol. 177(C).
    16. Hirose, Junichi & Kotani, Koji & Managi, Shunsuke, 2023. "Do autonomy and inquisitiveness contribute to SDGs? Implications from the matrilineal island of Palau," Economic Analysis and Policy, Elsevier, vol. 79(C), pages 303-318.
    17. Maria-Glòria Barberà-Mariné & Laura Fabregat-Aibar & Valeria Ferreira & Antonio Terceño, 2024. "One Step Away from 2030: An Assessment of the Progress of Sustainable Development Goals (SDGs) in the European Union [A un paso de 2030: Una evaluación del progreso de los Objetivos de Desarrollo S," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 36(6), pages 1372-1397, December.
    18. Mohsin Dhali & Shafiqul Hassan & Umashankar Subramaniam, 2023. "Comparative Analysis of Oil and Gas Legal Frameworks in Bangladesh and Nigeria: A Pathway towards Achieving Sustainable Energy through Policy," Sustainability, MDPI, vol. 15(21), pages 1-30, October.
    19. Gonzalo Hernández Soto, 2024. "Falling behind: Evaluating projected sustainable development goals progress across varied income countries," Sustainable Development, John Wiley & Sons, Ltd., vol. 32(3), pages 2194-2207, June.
    20. Dirk Schoenmaker & Hans Stegeman, 2023. "Can the Market Economy Deal with Sustainability?," De Economist, Springer, vol. 171(1), pages 25-49, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:130:y:2025:i:3:d:10.1007_s11192-025-05254-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.