IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v40y2013icp27-41.html
   My bibliography  Save this article

Evaluating a collaborative IT based research and development project

Author

Listed:
  • Khan, Zaheer
  • Ludlow, David
  • Caceres, Santiago

Abstract

In common with all projects, evaluating an Information Technology (IT) based research and development project is necessary in order to discover whether or not the outcomes of the project are successful. However, evaluating large-scale collaborative projects is especially difficult as: (i) stakeholders from different countries are involved who, almost inevitably, have diverse technological and/or application domain backgrounds and objectives; (ii) multiple and sometimes conflicting application specific and user-defined requirements exist; and (iii) multiple and often conflicting technological research and development objectives are apparent. In this paper, we share our experiences based on the large-scale integrated research project – The HUMBOLDT project – with project duration of 54 months, involving contributions from 27 partner organisations, plus 4 sub-contractors from 14 different European countries. In the HUMBOLDT project, a specific evaluation methodology was defined and utilised for the user evaluation of the project outcomes. The user evaluation performed on the HUMBOLDT Framework and its associated nine application scenarios from various application domains, resulted in not only an evaluation of the integrated project, but also revealed the benefits and disadvantages of the evaluation methodology. This paper presents the evaluation methodology, discusses in detail the process of applying it to the HUMBOLDT project and provides an in-depth analysis of the results, which can be usefully applied to other collaborative research projects in a variety of domains.

Suggested Citation

  • Khan, Zaheer & Ludlow, David & Caceres, Santiago, 2013. "Evaluating a collaborative IT based research and development project," Evaluation and Program Planning, Elsevier, vol. 40(C), pages 27-41.
    • Handle: RePEc:eee:epplan:v:40:y:2013:i:c:p:27-41
    DOI: 10.1016/j.evalprogplan.2013.04.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718913000347
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2013.04.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bryson, John M. & Patton, Michael Quinn & Bowman, Ruth A., 2011. "Working with evaluation stakeholders: A rationale, step-wise approach and toolkit," Evaluation and Program Planning, Elsevier, vol. 34(1), pages 1-12, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ullrich-French, Sarah & Cole, Amy N. & Montgomery, Anna K., 2016. "Evaluation development for a physical activity positive youth development program for girls," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 67-76.
    2. Yates, Brian T., 2021. "Toward collaborative cost-inclusive evaluation: Adaptations and transformations for evaluators and economists," Evaluation and Program Planning, Elsevier, vol. 89(C).
    3. Stella Pfisterer & Rob Van Tulder, 2020. "Navigating Governance Tensions to Enhance the Impact of Partnerships with the Private Sector for the SDGs," Sustainability, MDPI, vol. 13(1), pages 1-17, December.
    4. Gilbert Silvius & Ron Schipper, 2019. "Planning Project Stakeholder Engagement from a Sustainable Development Perspective," Administrative Sciences, MDPI, vol. 9(2), pages 1-22, June.
    5. Kivits, Robbert & Charles, Michael B., 2015. "Aviation planning policy in Australia: Identifying frames of reference to support public decision making," Journal of Air Transport Management, Elsevier, vol. 47(C), pages 102-111.
    6. Nam Phong Le & Thi Thu Phuong Nguyen & Dajian Zhu, 2018. "Understanding the Stakeholders’ Involvement in Utilizing Municipal Solid Waste in Agriculture through Composting: A Case Study of Hanoi, Vietnam," Sustainability, MDPI, vol. 10(7), pages 1-32, July.
    7. Wang, Jue & Aenis, Thomas & Hofmann-Souki, Susanne, 2018. "Triangulation in participation: Dynamic approaches for science-practice interaction in land-use decision making in rural China," Land Use Policy, Elsevier, vol. 72(C), pages 364-371.
    8. Lake, G. & Urban, M. & Giblin, F. & French, G. & Farrell, T., 2022. "Making a difference in the real world. User-centred impact evaluation of an eight-country, community-based early childhood programme," Evaluation and Program Planning, Elsevier, vol. 92(C).
    9. Ovidiu NICOLESCU & Ciprian NICOLESCU, 2020. "Company Relevant Stakeholders’ Responsibility: An Innovative Form Of Responsible Governance," Proceedings of the INTERNATIONAL MANAGEMENT CONFERENCE, Faculty of Management, Academy of Economic Studies, Bucharest, Romania, vol. 14(1), pages 319-334, November.
    10. Junwen Luo & Gonzalo Ordóñez-Matamoros & Stefan Kuhlmann, 2019. "The balancing role of evaluation mechanisms in organizational governance—The case of publicly funded research institutions," Research Evaluation, Oxford University Press, vol. 28(4), pages 344-354.
    11. Camilo Venegas & Andrea C. Sánchez-Alfonso & Crispín Celis & Fidson-Juarismy Vesga & Mauricio González Mendez, 2021. "Management Strategies and Stakeholders Analysis to Strengthen the Management and Use of Biosolids in a Colombian Municipality," Sustainability, MDPI, vol. 13(21), pages 1-25, November.
    12. Wendong Wu & Fang He & Taozhi Zhuang & Yuan Yi, 2020. "Stakeholder Analysis and Social Network Analysis in the Decision-Making of Industrial Land Redevelopment in China: The Case of Shanghai," IJERPH, MDPI, vol. 17(24), pages 1-27, December.
    13. Esposito, Giovanna & Freda, Maria Francesca, 2015. "Evaluating training context competence of use: Productive and unproductive models of use," Evaluation and Program Planning, Elsevier, vol. 50(C), pages 77-87.
    14. Arellano, Alexandra & Halsall, Tanya & Forneris, Tanya & Gaudet, Cindy, 2018. "Results of a utilization-focused evaluation of a Right To Play program for Indigenous youth," Evaluation and Program Planning, Elsevier, vol. 66(C), pages 156-164.
    15. Kingston, Kylie L. & Furneaux, Craig & de Zwaan, Laura & Alderman, Lyn, 2023. "Avoiding the accountability ‘sham-ritual’: An agonistic approach to beneficiaries’ participation in evaluation within nonprofit organisations," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 92(C).
    16. Weixuan Chen & Ali Cheshmehzangi & Eugenio Mangi & Timothy Heath & Changdong Ye & Ling Wang, 2022. "An Analysis of Residents’ Social Profiles Influencing Their Participation in Community Micro-Regeneration Projects in China: A Case Study of Yongtai Community, Guangzhou," Land, MDPI, vol. 11(6), pages 1-18, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:40:y:2013:i:c:p:27-41. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.