IDEAS home Printed from https://ideas.repec.org/p/zbw/fisidp/74.html
   My bibliography  Save this paper

Concepts and methods to measure societal impacts: An overview

Author

Listed:
  • Bührer, Susanne
  • Feidenheimer, Alexander
  • Walz, Rainer
  • Lindner, Ralf
  • Beckert, Bernd
  • Wallwaey, Elisa

Abstract

[Introduction] A critical analysis of the topic "societal impacts" is relevant and necessary for several reasons. These are: 1) The topic of societal impacts is currently at the top of the agenda in both (applied) science and in research and innovation (R&I) policy. 2) There is no uniform understanding of what a societal impact is in the literature. Definitions of societal impacts often refer to other impact dimensions (RRI, policy impacts, SSH impacts, cul-tural impacts, health impacts and sustainability indicators). 3) There is a great deal of confusion surrounding the topic of societal impacts. It has references to (1) impact measurement (Theory-based impact evaluation (TBIE), Theories of Change (ToCs), I-O-O-I (Input-Output-Outcome-Impact) models, impact pathways, participatory im-pact pathways, public value mapping, payback framework), (2) other discourses such as mis-sion orientation and addressing the Sustainable Development Goals (SDGs), and (3) questions concerning research assessment (responsible metrics, metrics tide, the Leiden manifesto). - This report addresses these challenges as follows: We present the different definitions of societal impacts in the next chapter (chapter 2.1) and the different strands of discourse that contribute to the societal impacts topic. We distinguish the discourses on societal impact in a narrower sense (chapter 2.2) from related discourses (chapter 2.3). The paper closes with some conclusions (chapter 3). Please note: This report concentrates on research funding and therefore excludes the societal im-pacts that (can) come about through sectoral policies. Second, following the logic of programme evaluations, we primarily adopt an ex post perspective, although we know that societal impacts (can) play an important role in ex ante impact assessments as well. Finally, it was not (yet) examined to what extent interaction effects occur between the different impact areas.

Suggested Citation

  • Bührer, Susanne & Feidenheimer, Alexander & Walz, Rainer & Lindner, Ralf & Beckert, Bernd & Wallwaey, Elisa, 2022. "Concepts and methods to measure societal impacts: An overview," Discussion Papers "Innovation Systems and Policy Analysis" 74, Fraunhofer Institute for Systems and Innovation Research (ISI).
    • Handle: RePEc:zbw:fisidp:74
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/262140/1/1811474128.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    2. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    3. Brian M. Belcher & Katherine E. Rasmussen & Matthew R. Kemshaw & Deborah A. Zornes, 2016. "Defining and assessing research quality in a transdisciplinary context," Research Evaluation, Oxford University Press, vol. 25(1), pages 1-17.
    4. Jack Spaapen & Leonie van Drooge, 2011. "Introducing ‘productive interactions’ in social impact assessment," Research Evaluation, Oxford University Press, vol. 20(3), pages 211-218, September.
    5. Emanuela Reale & Dragana Avramov & Kubra Canhial & Claire Donovan & Ramon Flecha & Poul Holm & Charles Larkin & Benedetto Lepori & Judith Mosoni-Fried & Esther Oliver & Emilia Primeri & Lidia Puigvert, 2018. "A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research," Research Evaluation, Oxford University Press, vol. 27(4), pages 298-308.
    6. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    7. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    8. Landry, Rejean & Amara, Nabil & Lamari, Moktar, 2001. "Utilization of social science research knowledge in Canada," Research Policy, Elsevier, vol. 30(2), pages 333-349, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matt, M. & Gaunand, A. & Joly, P-B. & Colinet, L., 2017. "Opening the black box of impact – Ideal-type impact pathways in a public agricultural research organization," Research Policy, Elsevier, vol. 46(1), pages 207-218.
    2. Andrea Bonaccorsi & Filippo Chiarello & Gualtiero Fantoni, 2021. "Impact for whom? Mapping the users of public research with lexicon-based text mining," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1745-1774, February.
    3. Alessandro Margherita & Gianluca Elia & Claudio Petti, 2022. "What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives," Sustainability, MDPI, vol. 14(5), pages 1-18, March.
    4. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.
    5. Jorrit P Smit & Laurens K Hessels, 2021. "The production of scientific and societal value in research evaluation: a review of societal impact assessment methods [Systems Thinking, Knowledge and Action: Towards Better Models and Methods]," Research Evaluation, Oxford University Press, vol. 30(3), pages 323-335.
    6. A. Gaunand & L. Colinet & P.-B. Joly & M. Matt, 2022. "Counting what really counts? Assessing the political impact of science," The Journal of Technology Transfer, Springer, vol. 47(3), pages 699-721, June.
    7. Irwin Feller, 2022. "Assessing the societal impact of publicly funded research," The Journal of Technology Transfer, Springer, vol. 47(3), pages 632-650, June.
    8. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    9. Pierre Benoit Joly & Laurence Colinet & Ariane Gaunand & Stephane Lemarié & Mireille Matt, 2016. "Agricultural research impact assessment: issues, methods and challenges," Working Papers hal-01431457, HAL.
    10. Pierre-Benoit Joly & Mireille Matt, 2022. "Towards a new generation of research impact assessment approaches," The Journal of Technology Transfer, Springer, vol. 47(3), pages 621-631, June.
    11. Ciarli, Tommaso & Ràfols, Ismael, 2019. "The relation between research priorities and societal demands: The case of rice," Research Policy, Elsevier, vol. 48(4), pages 949-967.
    12. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    13. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    14. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    15. Lutz Bornmann, 2020. "Bibliometrics-based decision tree (BBDT) for deciding whether two universities in the Leiden ranking differ substantially in their performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1255-1258, February.
    16. Han Zheng & L. G. Pee & Dan Zhang, 2021. "Societal impact of research: a text mining study of impact types," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7397-7417, September.
    17. Stina Hansson & Merritt Polk, 2018. "Assessing the impact of transdisciplinary research: The usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact," Research Evaluation, Oxford University Press, vol. 27(2), pages 132-144.
    18. Alba Viana Lora & Marta Gemma Nel-lo Andreu, 2020. "Alternative Metrics for Assessing the Social Impact of Tourism Research," Sustainability, MDPI, vol. 12(10), pages 1-12, May.
    19. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    20. Ole Henning Sørensen & Jakob Bjørner & Andreas Holtermann & Johnny Dyreborg & Jorid Birkelund Sørli & Jesper Kristiansen & Steffen Bohni Nielsen, 2022. "Measuring societal impact of research—Developing and validating an impact instrument for occupational health and safety," Research Evaluation, Oxford University Press, vol. 31(1), pages 118-131.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:fisidp:74. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/isfhgde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.