IDEAS home Printed from https://ideas.repec.org/p/hal/wpaper/hal-01431457.html
   My bibliography  Save this paper

Agricultural research impact assessment: issues, methods and challenges

Author

Listed:
  • Pierre Benoit Joly

    (LISIS - Laboratoire Interdisciplinaire Sciences, Innovations, Sociétés - INRA - Institut National de la Recherche Agronomique - UPEM - Université Paris-Est Marne-la-Vallée - ESIEE Paris - CNRS - Centre National de la Recherche Scientifique)

  • Laurence Colinet

    (CODIR - Collège de Direction - INRA - Institut National de la Recherche Agronomique)

  • Ariane Gaunand

    (Délégation à l'évaluation - INRA - Institut National de la Recherche Agronomique)

  • Stephane Lemarié

    (GAEL - Laboratoire d'Economie Appliquée de Grenoble - Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology - INRA - Institut National de la Recherche Agronomique - CNRS - Centre National de la Recherche Scientifique - UGA [2016-2019] - Université Grenoble Alpes [2016-2019])

  • Mireille Matt

    (GAEL - Laboratoire d'Economie Appliquée de Grenoble - Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology - INRA - Institut National de la Recherche Agronomique - CNRS - Centre National de la Recherche Scientifique - UGA [2016-2019] - Université Grenoble Alpes [2016-2019])

Abstract

The Research Impact Assessment (RIA) is expected to increase the efficiency with which public funds are used, and to improve more broadly the functioning of the research and innovation system and its contribution to address a wide range of socio-economic and environmental issues. Both standard economic approaches, which aim to estimate the economic benefits of research investments, and case-study approaches, which aim to analyse the processes of impact generation, have been applied to agricultural research in practice. Standard economic approaches generally focus on public research as information on private efforts in agricultural research is limited, and on economic impacts such as productivity growth. Case studies provide richer information, through a narrative, and highlight the complex relationships among the various variables, events and actors, but it is difficult to standardise results and scale them up. The challenge for RIA is to take into account broader impacts that go beyond science and economic impacts, and to improve knowledge on impact-generating mechanisms. This has become more difficult as agricultural research and innovation systems are increasingly open and complex, and changing quickly. Observation of practices applied to agricultural research in five selected organisations confirms the difference found in RIA between academic research and in practice. In both, the assessment systems pursue the same objectives: 1) Learning: enhance the know-how to produce an environment conducive to socio-economic impact; 2) Capacity building: spread the culture of socio-economic impact to its researchers; and 3) Reporting to stakeholders: from accountability purposes to advocacy targeted to various audiences. The accountability objective, including estimating returns on the financial investment, poses complex challenges and is in tension with the learning and capacity building objectives. The future of RIA will depend on the capacity to improve estimation methods and gather quality information (which also takes into account non-economic impacts) and the sharing of good practices.

Suggested Citation

  • Pierre Benoit Joly & Laurence Colinet & Ariane Gaunand & Stephane Lemarié & Mireille Matt, 2016. "Agricultural research impact assessment: issues, methods and challenges," Working Papers hal-01431457, HAL.
    • Handle: RePEc:hal:wpaper:hal-01431457
    DOI: 10.1787/18156797
    Note: View the original document on HAL open archive server: https://hal.science/hal-01431457
    as

    Download full text from publisher

    File URL: https://hal.science/hal-01431457/document
    Download Restriction: no
    File URL: https://libkey.io/10.1787/18156797?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Alston, Julian M. & Wyatt, T. J. & Pardey, Philip G. & Marra, Michele C. & Chan-Kang, Connie, 2000. "A meta-analysis of rates of return to agricultural R & D: ex pede Herculem?," Research reports 113, International Food Policy Research Institute (IFPRI).
    2. Salter, Ammon J. & Martin, Ben R., 2001. "The economic benefits of publicly funded basic research: a critical review," Research Policy, Elsevier, vol. 30(3), pages 509-532, March.
    3. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    4. Henderson, Rebecca M. & Newell, Richard G. (ed.), 2011. "Accelerating Energy Innovation," National Bureau of Economic Research Books, University of Chicago Press, number 9780226326832, December.
    5. Jordi Molas-Gallart & Puay Tang, 2011. "Tracing ‘productive interactions’ to identify social impacts: an example from the social sciences," Research Evaluation, Oxford University Press, vol. 20(3), pages 219-226, September.
    6. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    7. Stefan de Jong & Katharine Barker & Deborah Cox & Thordis Sveinsdottir & Peter Van den Besselaar, 2014. "Understanding societal impact through productive interactions: ICT research as a case," Research Evaluation, Oxford University Press, vol. 23(2), pages 89-102.
    8. Claire Donovan, 2007. "The qualitative future of research evaluation," Science and Public Policy, Oxford University Press, vol. 34(8), pages 585-597, October.
    9. Patricio Grassini & Kent M. Eskridge & Kenneth G. Cassman, 2013. "Distinguishing between yield advances and yield plateaus in historical crop production trends," Nature Communications, Nature, vol. 4(1), pages 1-11, December.
    10. Gladys Mutangadura & George W. Norton, 1999. "Agricultural research priority setting under multiple objectives: an example from Zimbabwe," Agricultural Economics, International Association of Agricultural Economists, vol. 20(3), pages 277-286, May.
    11. Catherine Moreddu, 2016. "Public-Private Partnerships for Agricultural Innovation: Lessons From Recent Experiences," OECD Food, Agriculture and Fisheries Papers 92, OECD Publishing.
    12. Zvi Griliches, 1958. "Research Costs and Social Returns: Hybrid Corn and Related Innovations," Journal of Political Economy, University of Chicago Press, vol. 66, pages 419-419.
    13. Mowery, David C. & Nelson, Richard R. & Martin, Ben R., 2010. "Technology policy and global warming: Why new policy models are needed (or why putting new wine in old bottles won't work)," Research Policy, Elsevier, vol. 39(8), pages 1011-1023, October.
    14. Scherer, F. M. & Harhoff, Dietmar, 2000. "Technology policy for a world of skew-distributed outcomes," Research Policy, Elsevier, vol. 29(4-5), pages 559-566, April.
    15. Evenson, Robert E., 2001. "Economic impacts of agricultural research and extension," Handbook of Agricultural Economics, in: B. L. Gardner & G. C. Rausser (ed.), Handbook of Agricultural Economics, edition 1, volume 1, chapter 11, pages 573-628, Elsevier.
    16. Douthwaite, Boru & Kuby, Thomas & van de Fliert, Elske & Schulz, Steffen, 2003. "Impact pathway evaluation: an approach for achieving and attributing impact in complex systems," Agricultural Systems, Elsevier, vol. 78(2), pages 243-265, November.
    17. Sparger, John Adam & Norton, George W. & Heisey, Paul W. & Alwang, Jeffrey, 2013. "Is the share of agricultural maintenance research rising in the United States?," Food Policy, Elsevier, vol. 38(C), pages 126-135.
    18. Sheng, Yu & Gray, Emily M. & Mullen, John D., 2011. "Public investment in R&D and extension and productivity in Australian broadacre agriculture," 2011 Conference (55th), February 8-11, 2011, Melbourne, Australia 100712, Australian Agricultural and Resource Economics Society.
    19. Foray, D. & Mowery, D.C. & Nelson, R.R., 2012. "Public R&D and social challenges: What lessons from mission R&D programs?," Research Policy, Elsevier, vol. 41(10), pages 1697-1702.
    20. Terrance M. Hurley & Xudong Rao & Philip G. Pardey, 2014. "Re-examining the Reported Rates of Return to Food and Agricultural Research and Development," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 96(5), pages 1492-1504.
    21. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    22. Andersen, Matthew A., 2015. "Public investment in U.S. agricultural R&D and the economic benefits," Food Policy, Elsevier, vol. 51(C), pages 38-43.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alessandro Magrini & Fabio Bartolini & Alessandra Coli & Barbara Pacini, 2019. "A structural equation model to assess the impact of agricultural research expenditure on multiple dimensions," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(4), pages 2063-2080, July.
    2. Markow, Jekaterina & Fieldsend, Andrew F. & Münchhausen, Susanne von & Häring, Anna Maria, 2023. "Building agricultural innovation capacity from the bottom up: Using spillover effects from projects to strengthen agricultural innovation systems," Agricultural Systems, Elsevier, vol. 209(C).
    3. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    4. André Martinuzzi & Markus Hametner & Andreas Windsperger & Nadine Brunnhuber, 2023. "A Framework for Assessing the Climate Impacts of Research and Innovation Projects and Programmes," Sustainability, MDPI, vol. 15(24), pages 1-24, December.
    5. Dominique Barjolle & Peter Midmore & Otto Schmid, 2018. "Tracing the Pathways from Research to Innovation: Evidence from Case Studies," EuroChoices, The Agricultural Economics Society, vol. 17(1), pages 11-18, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    2. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    3. Stéphane Lemarié & Valérie Orozco & Jean-Pierre Butault & Antonio Musolesi & Michel Simioni & Bertrand Schmitt, 2020. "Assessing the long-term impact of agricultural research on productivity: evidence from France," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 47(4), pages 1559-1586.
    4. Matt, M. & Gaunand, A. & Joly, P-B. & Colinet, L., 2017. "Opening the black box of impact – Ideal-type impact pathways in a public agricultural research organization," Research Policy, Elsevier, vol. 46(1), pages 207-218.
    5. Pierre-Benoit Joly & Mireille Matt, 2022. "Towards a new generation of research impact assessment approaches," The Journal of Technology Transfer, Springer, vol. 47(3), pages 621-631, June.
    6. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    7. Andrea Bonaccorsi & Filippo Chiarello & Gualtiero Fantoni, 2021. "Impact for whom? Mapping the users of public research with lexicon-based text mining," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1745-1774, February.
    8. Gaunand, A. & Hocdé, A. & Lemarié, S. & Matt, M. & Turckheim, E.de, 2015. "How does public agricultural research impact society? A characterization of various patterns," Research Policy, Elsevier, vol. 44(4), pages 849-861.
    9. Alessandro Magrini & Fabio Bartolini & Alessandra Coli & Barbara Pacini, 2019. "A structural equation model to assess the impact of agricultural research expenditure on multiple dimensions," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(4), pages 2063-2080, July.
    10. Gunnar Sivertsen & Ingeborg Meijer, 2020. "Normal versus extraordinary societal impact: how to understand, evaluate, and improve research activities in their relations to society?," Research Evaluation, Oxford University Press, vol. 29(1), pages 66-70.
    11. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    12. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.
    13. Maredia, Mywish K. & Raitzer, David A., 2012. "Review and analysis of documented patterns of agricultural research impacts in Southeast Asia," Agricultural Systems, Elsevier, vol. 106(1), pages 46-58.
    14. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    15. Winskel, Mark & Radcliffe, Jonathan & Skea, Jim & Wang, Xinxin, 2014. "Remaking the UK's energy technology innovation system: From the margins to the mainstream," Energy Policy, Elsevier, vol. 68(C), pages 591-602.
    16. Archibugi, Daniele & Filippetti, Andrea, 2018. "The retreat of public research and its adverse consequences on innovation," Technological Forecasting and Social Change, Elsevier, vol. 127(C), pages 97-111.
    17. Faure, Guy & Barret, Danielle & Blundo-Canto, Genowefa & Dabat, Marie-Hélène & Devaux-Spatarakis, Agathe & Le Guerroué, Jean Louis & Marquié, Catherine & Mathé, Syndhia & Temple, Ludovic & Toillier, A, 2018. "How different agricultural research models contribute to impacts: Evidence from 13 case studies in developing countries," Agricultural Systems, Elsevier, vol. 165(C), pages 128-136.
    18. Andrea Borsato & Andre Lorentz, 2023. "Open Science vs. Mission-oriented Policies and the Long-run Dynamics of Integrated Economies: An Agent-based Model with a Kaldorian Flavour," Working Papers of BETA 2023-17, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    19. Toole, Andrew A., 2012. "The impact of public basic research on industrial innovation: Evidence from the pharmaceutical industry," Research Policy, Elsevier, vol. 41(1), pages 1-12.
    20. Hurley, Terrance M. & Pardey, Philip G. & Rao, Xudong & Andrade, Robert S., 2016. "Returns to Food and Agricultural R&D Investments Worldwide, 1958-2015," Briefs 249356, University of Minnesota, International Science and Technology Practice and Policy.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:wpaper:hal-01431457. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.