IDEAS home Printed from https://ideas.repec.org/a/zna/indecs/v18y2020i4p449-458.html
   My bibliography  Save this article

Innovative strategies for creating and assessing research quality and societal impact in social sciences and humanities

Author

Listed:
  • Jovana Janinovic

    (University of Montenegro, Podgorica, Montenegro)

  • Sanja Pekovic

    (University of Montenegro, Kotor, Montenegro)

  • Dijana Vuckovic

    (University of Montenegro, Podgorica, Montenegro)

  • Stevo Popovic

    (University of Montenegro, Podgorica, Montenegro)

  • Rajka Djokovic

    (University of Montenegro, Podgorica, Montenegro)

  • Mirjana Pejiæ Bach

    (University of Zagreb - Faculty of Economics and Business, Zagreb, Croatia)

Abstract

Contemporary academic evaluation regimes, aiming to quantify and rank research quality, scholars and institutions, mostly rely on criteria such as metrics and citation scores. However, researchers started questioning recently whether these are the most appropriate instruments to evaluate academic excellence, especially in social sciences and humanities. Predominant evaluation criteria, such as peer review and citation index reflect only a limited scope of the research quality and relevance, as many dimensions cannot be seized through these traditional indicators, such as societal impact. In response to this criticism, a number of innovative research quality evaluation methods emerged in recent years, aiming to better reflect the complexity of often transdisciplinary research in social sciences and humanities. In this article, we attempt to review some of the most innovative strategies for assessing research quality, especially concerning transdisciplinary research in social sciences and humanities, as well as the societal impact and 'productive interactions'. We conclude that while other innovative strategies are still in the pioneer phase, requiring more research and practical implementation, measuring societal impact emerged as a reliable, relevant, comprehensive and applicable strategy for evaluating research quality in social sciences and humanities.

Suggested Citation

  • Jovana Janinovic & Sanja Pekovic & Dijana Vuckovic & Stevo Popovic & Rajka Djokovic & Mirjana Pejiæ Bach, 2020. "Innovative strategies for creating and assessing research quality and societal impact in social sciences and humanities," Interdisciplinary Description of Complex Systems - scientific journal, Croatian Interdisciplinary Society Provider Homepage: http://indecs.eu, vol. 18(4), pages 449-458.
  • Handle: RePEc:zna:indecs:v:18:y:2020:i:4:p:449-458
    as

    Download full text from publisher

    File URL: http://indecs.eu/2020/indecs2020-pp449-458.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Stefan P. L. de Jong & Jorrit Smit & Leonie van Drooge, 2016. "Scientists’ response to societal impact policies: A policy paradox," Science and Public Policy, Oxford University Press, vol. 43(1), pages 102-114.
    2. Pablo D’Este & Puay Tang & Surya Mahdi & Andy Neely & Mabel Sánchez-Barrioluengo, 2013. "The pursuit of academic excellence and business engagement: is it irreconcilable?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 481-502, May.
    3. Irwin Feller, 2006. "Multiple actors, multiple settings, multiple criteria: issues in assessing interdisciplinary research," Research Evaluation, Oxford University Press, vol. 15(1), pages 5-15, April.
    4. Ramon Flecha & Marta Soler-Gallart & Teresa Sordé, 2015. "Europe must fund social sciences," Nature, Nature, vol. 528(7581), pages 193-193, December.
    5. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    6. Stefan de Jong & Katharine Barker & Deborah Cox & Thordis Sveinsdottir & Peter Van den Besselaar, 2014. "Understanding societal impact through productive interactions: ICT research as a case," Research Evaluation, Oxford University Press, vol. 23(2), pages 89-102.
    7. Veronica Boix Mansilla & Irwin Feller & Howard Gardner, 2006. "Quality assessment in interdisciplinary research and education," Research Evaluation, Oxford University Press, vol. 15(1), pages 69-74, April.
    8. Zoe Bulaitis, 2017. "Measuring impact in the humanities: Learning from accountability and economics in a contemporary history of cultural value," Palgrave Communications, Palgrave Macmillan, vol. 3(1), pages 1-11, December.
    9. Pablo Jensen & Jean-Baptiste Rouquier & Pablo Kreimer & Yves Croissant, 2008. "Scientists who engage with society perform better academically," Science and Public Policy, Oxford University Press, vol. 35(7), pages 527-541, August.
    10. Gaby Haddow & Björn Hammarfelt, 2019. "Quality, impact, and quantification: Indicators and metrics use by social scientists," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(1), pages 16-26, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Juha-Pekka Lauronen, 2022. "The epistemic, production, and accountability prospects of social impact: An analysis of strategic research proposals," Research Evaluation, Oxford University Press, vol. 31(2), pages 214-225.
    2. Lin Zhang & Gunnar Sivertsen & Huiying Du & Ying Huang & Wolfgang Glänzel, 2021. "Gender differences in the aims and impacts of research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 8861-8886, November.
    3. Zhang, Lin & Sivertsen, Gunnar & Du, Huiying & HUANG, Ying & Glänzel, Wolfgang, 2021. "Gender differences in the aims and impacts of research," SocArXiv 9n347, Center for Open Science.
    4. Jorrit P Smit & Laurens K Hessels, 2021. "The production of scientific and societal value in research evaluation: a review of societal impact assessment methods [Systems Thinking, Knowledge and Action: Towards Better Models and Methods]," Research Evaluation, Oxford University Press, vol. 30(3), pages 323-335.
    5. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    6. Reetta Muhonen & Paul Benneworth & Julia Olmos-Peñuela, 2020. "From productive interactions to impact pathways: Understanding the key dimensions in developing SSH research societal impact," Research Evaluation, Oxford University Press, vol. 29(1), pages 34-47.
    7. Jonathan P. Doh & Lorraine Eden & Anne S. Tsui & Srilata Zaheer, 2023. "Developing international business scholarship for global societal impact," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 54(5), pages 757-767, July.
    8. Stefan P L de Jong & Corina Balaban & Maria Nedeva, 2022. "From ‘productive interactions’ to ‘enabling conditions’: The role of organizations in generating societal impact of academic research [One Size Does Not Fit All! New Perspectives on the University ," Science and Public Policy, Oxford University Press, vol. 49(4), pages 643-645.
    9. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    10. Froese, Anna & Woiwode, Hendrik & Suckow, Silvio, 2019. "Mission Impossible? Neue Wege zu Interdisziplinarität: Empfehlungen für Wissenschaft, Wissenschaftspolitik und Praxis," Discussion Papers, Research Group Science Policy Studies SP III 2019-601, WZB Berlin Social Science Center.
    11. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    12. Stina Hansson & Merritt Polk, 2018. "Assessing the impact of transdisciplinary research: The usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact," Research Evaluation, Oxford University Press, vol. 27(2), pages 132-144.
    13. Alba Viana Lora & Marta Gemma Nel-lo Andreu, 2020. "Alternative Metrics for Assessing the Social Impact of Tourism Research," Sustainability, MDPI, vol. 12(10), pages 1-12, May.
    14. Helka Kalliomäki & Sampo Ruoppila & Jenni Airaksinen, 2021. "It takes two to tango: Examining productive interactions in urban research collaboration [Generating Research Questions through Problematization]," Research Evaluation, Oxford University Press, vol. 30(4), pages 529-539.
    15. D’Este, Pablo & Robinson-García, Nicolás, 2023. "Interdisciplinary research and the societal visibility of science: The advantages of spanning multiple and distant scientific fields," Research Policy, Elsevier, vol. 52(2).
    16. Junwen Luo & Lai Ma & Kalpana Shankar, 2021. "Does the inclusion of non-academic reviewers make any difference for grant impact panels? [Understanding the Long Term Impact of the Framework Programme, European Policy Evaluation Consortium (EPEC," Science and Public Policy, Oxford University Press, vol. 48(6), pages 763-775.
    17. Hans Jonker & Florian Vanlee & Walter Ysebaert, 2022. "Societal impact of university research in the written press: media attention in the context of SIUR and the open science agenda among social scientists in Flanders, Belgium," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7289-7306, December.
    18. Andrea Bonaccorsi & Nicola Melluso & Francesco Alessandro Massucci, 2022. "Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6961-6991, December.
    19. Marra, Mita, 2022. "Productive interactions in digital training partnerships: Lessons learned for regional development and university societal impact assessment," Evaluation and Program Planning, Elsevier, vol. 95(C).
    20. Matt, M. & Gaunand, A. & Joly, P-B. & Colinet, L., 2017. "Opening the black box of impact – Ideal-type impact pathways in a public agricultural research organization," Research Policy, Elsevier, vol. 46(1), pages 207-218.

    More about this item

    Keywords

    research evaluation; research quality; evaluative inquiry; societal impact;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zna:indecs:v:18:y:2020:i:4:p:449-458. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Josip Stepanic (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.