IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v30y2021i4p458-469..html
   My bibliography  Save this article

Looking for evidence of research impact and use: A qualitative study of an Australian research-policy system
[Public Social Policy Development and Implementation: A Case Study of the Ghana National Health Insurance Scheme]

Author

Listed:
  • Robyn S. Newson
  • Lucie Rychetnik
  • Lesley King
  • Andrew J. Milat
  • Adrian E. Bauman

Abstract

Current assessments of research impact have been criticized for capturing what can be easily counted not what actually counts. To empirically examine this issue, we approached measuring research impact from two directions, tracing forwards from research and backwards from policy, within a defined research-policy system (childhood obesity prevention research and policy in New South Wales, Australia from 2000 to 2015). The forward tracing research impact assessment component traced a sample of 148 local research projects forward to examine their policy impacts. Of the projects considered, 16% had an impact on local policy and for a further 19%, decision-makers were aware of the research, but there was no evidence it influenced policy decisions. The backward tracing component of the study included an analysis of research use across three policy initiatives. It provided a more nuanced understanding of the relative influence of research on policy. Both direct uses of specific research and indirect uses of research incorporated as broader bodies of knowledge were evident. Measuring research impact from both directions captured the diverse ways that research was used in decision-making. Our findings illustrate complexities in the assessment process and in real-life policymaking trajectories. They highlight the role that timing of assessment plays in perception of impacts and difficulties attributing longer-term impacts to specific research. This study supports the use of models where politics and complex system dynamics shape knowledge and its influence on decision-making, rather than research being the primary driver for policy change.

Suggested Citation

  • Robyn S. Newson & Lucie Rychetnik & Lesley King & Andrew J. Milat & Adrian E. Bauman, 2021. "Looking for evidence of research impact and use: A qualitative study of an Australian research-policy system [Public Social Policy Development and Implementation: A Case Study of the Ghana National," Research Evaluation, Oxford University Press, vol. 30(4), pages 458-469.
    • Handle: RePEc:oup:rseval:v:30:y:2021:i:4:p:458-469.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvab017
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shelley Bowen & Anthony B Zwi, 2005. "Pathways to “Evidence-Informed” Policy and Practice: A Framework for Action," PLOS Medicine, Public Library of Science, vol. 2(7), pages 1-1, May.
    2. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefan P L de Jong & Corina Balaban & Maria Nedeva, 2022. "From ‘productive interactions’ to ‘enabling conditions’: The role of organizations in generating societal impact of academic research [One Size Does Not Fit All! New Perspectives on the University ," Science and Public Policy, Oxford University Press, vol. 49(4), pages 643-645.
    2. Rosa Kuipers-Dirven & Matthijs Janssen & Jarno Hoekman, 2023. "Assessing university policies for enhancing societal impact of academic research: A multicriteria mapping approach," Research Evaluation, Oxford University Press, vol. 32(2), pages 371-383.
    3. Conor O’Kane & Jing A. Zhang & Jarrod Haar & James A. Cunningham, 2023. "How scientists interpret and address funding criteria: value creation and undesirable side effects," Small Business Economics, Springer, vol. 61(2), pages 799-826, August.
    4. Amara, Nabil & Olmos-Peñuela, Julia & Fernández-de-Lucio, Ignacio, 2019. "Overcoming the “lost before translation” problem: An exploratory study," Research Policy, Elsevier, vol. 48(1), pages 22-36.
    5. Adrian Rauchfleisch & Mike S Schäfer & Dario Siegen, 2021. "Beyond the ivory tower: Measuring and explaining academic engagement with journalists, politicians and industry representatives among Swiss professorss," PLOS ONE, Public Library of Science, vol. 16(5), pages 1-20, May.
    6. Ole Henning Sørensen & Jakob Bjørner & Andreas Holtermann & Johnny Dyreborg & Jorid Birkelund Sørli & Jesper Kristiansen & Steffen Bohni Nielsen, 2022. "Measuring societal impact of research—Developing and validating an impact instrument for occupational health and safety," Research Evaluation, Oxford University Press, vol. 31(1), pages 118-131.
    7. David Barberá-Tomás & Joaquín M. Azagra-Caro & Pablo D’Este, 2022. "Dynamic perspectives on technology transfer: introduction to the special section," The Journal of Technology Transfer, Springer, vol. 47(5), pages 1299-1307, October.
    8. Frank J. Rijnsoever & Laurens K. Hessels, 2021. "How academic researchers select collaborative research projects: a choice experiment," The Journal of Technology Transfer, Springer, vol. 46(6), pages 1917-1948, December.
    9. Lin Zhang & Gunnar Sivertsen & Huiying Du & Ying Huang & Wolfgang Glänzel, 2021. "Gender differences in the aims and impacts of research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 8861-8886, November.
    10. Thomas, Duncan Andrew & Ramos-Vielba, Irene, 2022. "Reframing study of research(er) funding towards configurations and trails," SocArXiv uty2v, Center for Open Science.
    11. Junwen Luo & Lai Ma & Kalpana Shankar, 2021. "Does the inclusion of non-academic reviewers make any difference for grant impact panels? [Understanding the Long Term Impact of the Framework Programme, European Policy Evaluation Consortium (EPEC," Science and Public Policy, Oxford University Press, vol. 48(6), pages 763-775.
    12. Suominen, Arho & Kauppinen, Henni & Hyytinen, Kirsi, 2021. "‘Gold’, ‘Ribbon’ or ‘Puzzle’: What motivates researchers to work in Research and Technology Organizations," Technological Forecasting and Social Change, Elsevier, vol. 170(C).
    13. Wesley M. Cohen & Henry Sauermann & Paula Stephan, 2020. "Not in the Job Description: The Commercial Activities of Academic Scientists and Engineers," Management Science, INFORMS, vol. 66(9), pages 4108-4117, September.
    14. Zhang, Lin & Sivertsen, Gunnar & Du, Huiying & HUANG, Ying & Glänzel, Wolfgang, 2021. "Gender differences in the aims and impacts of research," SocArXiv 9n347, Center for Open Science.
    15. Lars K. Hallstrom & Glen T. Hvenegaard, 2021. "Fostering Evidence-Informed Decision-Making for Protected Areas through the Alberta Parks Social Science Working Group," Land, MDPI, vol. 10(2), pages 1-15, February.
    16. Pandey, Poonam & Pansera, Mario, 2020. "Bringing Laxmi and Saraswati together: Nano-scientists and academic entrepreneurship in India," Technology in Society, Elsevier, vol. 63(C).
    17. Chams, Nour & Guesmi, Bouali & Gil, Jose M. & Molins, Mireia & Cubel, Rosa, 2021. "Between “Research Producers” and “Research Adopters”: The Role of Knowledge and Innovation Transfer on Sustainability Impact," 2021 Conference, August 17-31, 2021, Virtual 315264, International Association of Agricultural Economists.
    18. Yitong Wang & Tingting Qiu & Junwen Zhou & Clément Francois & Mondher Toumi, 2021. "Which Criteria are Considered and How are They Evaluated in Health Technology Assessments? A Review of Methodological Guidelines Used in Western and Asian Countries," Applied Health Economics and Health Policy, Springer, vol. 19(3), pages 281-304, May.
    19. Gunnar Sivertsen & Ingeborg Meijer, 2020. "Normal versus extraordinary societal impact: how to understand, evaluate, and improve research activities in their relations to society?," Research Evaluation, Oxford University Press, vol. 29(1), pages 66-70.
    20. Igors Skute, 2019. "Opening the black box of academic entrepreneurship: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 237-265, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:30:y:2021:i:4:p:458-469.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.