IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v55y2016icp103-111.html

An evaluation capacity building toolkit for principal investigators of undergraduate research experiences: A demonstration of transforming theory into practice

Author

Listed:
  • Rorrer, Audrey S.

Abstract

This paper describes the approach and process undertaken to develop evaluation capacity among the leaders of a federally funded undergraduate research program. An evaluation toolkit was developed for Computer and Information Sciences and Engineering11Computer and Information Sciences and Engineering, CISE. Research Experiences for Undergraduates22Research Experiences for Undergraduates, REU. (CISE REU) programs to address the ongoing need for evaluation capacity among principal investigators who manage program evaluation. The toolkit was the result of collaboration within the CISE REU community with the purpose being to provide targeted instructional resources and tools for quality program evaluation. Challenges were to balance the desire for standardized assessment with the responsibility to account for individual program contexts. Toolkit contents included instructional materials about evaluation practice, a standardized applicant management tool, and a modulated outcomes measure. Resulting benefits from toolkit deployment were having cost effective, sustainable evaluation tools, a community evaluation forum, and aggregate measurement of key program outcomes for the national program. Lessons learned included the imperative of understanding the evaluation context, engaging stakeholders, and building stakeholder trust. Results from project measures are presented along with a discussion of guidelines for facilitating evaluation capacity building that will serve a variety of contexts.

Suggested Citation

  • Rorrer, Audrey S., 2016. "An evaluation capacity building toolkit for principal investigators of undergraduate research experiences: A demonstration of transforming theory into practice," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 103-111.
    • Handle: RePEc:eee:epplan:v:55:y:2016:i:c:p:103-111
    DOI: 10.1016/j.evalprogplan.2015.12.006
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718915001330
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2015.12.006?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Hardré, Patricia L. & Slater, Janis & Nanny, Mark, 2010. "Redesigning and aligning assessment and evaluation for a federally funded math and science teacher educational program," Evaluation and Program Planning, Elsevier, vol. 33(4), pages 498-510, November.
    2. Bourgeois, Isabelle & Whynot, Jane & Thériault, Étienne, 2015. "Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned," Evaluation and Program Planning, Elsevier, vol. 50(C), pages 47-55.
    3. Patton, Michael Quinn & Horton, Douglas, 2008. "Utilization-focused evaluation for agricultural innovation," ILAC Briefs 52533, Institutional Learning and Change (ILAC) Initiative.
    4. DeGroff, Amy & Schooley, Michael & Chapel, Thomas & Poister, Theodore H., 2010. "Challenges and strategies in applying performance measurement to federal public health programs," Evaluation and Program Planning, Elsevier, vol. 33(4), pages 365-372, November.
    5. Campbell, Rebecca & Townsend, Stephanie M. & Shaw, Jessica & Karim, Nidal & Markowitz, Jenifer, 2015. "Can a workbook work? Examining whether a practitioner evaluation toolkit can promote instrumental use," Evaluation and Program Planning, Elsevier, vol. 52(C), pages 107-117.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abdul-Manan, Amir F.N. & Baharuddin, Azizan & Chang, Lee Wei, 2015. "Application of theory-based evaluation for the critical analysis of national biofuel policy: A case study in Malaysia," Evaluation and Program Planning, Elsevier, vol. 52(C), pages 39-49.
    2. Romero-Gutierrez, Miguel & Jimenez-Liso, M. Rut & Martinez-Chico, Maria, 2016. "SWOT analysis to evaluate the programme of a joint online/onsite master's degree in environmental education through the students’ perceptions," Evaluation and Program Planning, Elsevier, vol. 54(C), pages 41-49.
    3. Ofek, Yuval, 2017. "Evaluating social exclusion interventions in university-community partnerships," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 46-55.
    4. Martens, Krystin S.R., 2018. "How program evaluators use and learn to use rubrics to make evaluative reasoning explicit," Evaluation and Program Planning, Elsevier, vol. 69(C), pages 25-32.
    5. Telch, Fabian, 2025. "Understanding how national development planning (NDP) shapes public institutions and procedures for development: the case of Colombia," World Development Perspectives, Elsevier, vol. 39(C).
    6. Bourgeois, Isabelle & Whynot, Jane, 2018. "The influence of evaluation recommendations on instrumental and conceptual uses: A preliminary analysis," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 13-18.
    7. Lifshitz, Chen Chana, 2017. "Fostering employability among youth at-risk in a multi-cultural context: Insights from a pilot intervention program," Children and Youth Services Review, Elsevier, vol. 76(C), pages 20-34.
    8. LaVelle, John M. & Davies, Randall, 2021. "Seeking consensus: Defining foundational concepts for a graduate level introductory program evaluation course," Evaluation and Program Planning, Elsevier, vol. 88(C).
    9. Melz, Heidi & Fromknecht, Anne E. & Masters, Loren D. & Richards, Tammy & Sun, Jing, 2023. "Incorporating multiple data sources to assess changes in organizational capacity in child welfare systems," Evaluation and Program Planning, Elsevier, vol. 97(C).
    10. Maria de Fátima Souza & Ana Carvalho, 2018. "An Organizational Capacity model for wine cooperatives," NIPE Working Papers 10/2018, NIPE - Universidade do Minho.
    11. Healy, John & Hughes, Jeffrey & Donnelly-Cox, Gemma & Shantz, Amanda, 2024. "A long and winding road: The hard graft of scaling social change in complex systems," Journal of Business Venturing Insights, Elsevier, vol. 21(C).
    12. Wingate, Lori A. & Smith, Nick L. & Perk, Emma, 2018. "The project vita: A dynamic knowledge management tool," Evaluation and Program Planning, Elsevier, vol. 71(C), pages 22-27.
    13. Metta, Matteo & Ciliberti, Stefano & Obi, Chinedu & Bartolini, Fabio & Klerkx, Laurens & Brunori, Gianluca, 2022. "An integrated socio-cyber-physical system framework to assess responsible digitalisation in agriculture: A first application with Living Labs in Europe," Agricultural Systems, Elsevier, vol. 203(C).
    14. Mikko V. Pohjola & Pasi Pohjola & Marko Tainio & Jouni T. Tuomisto, 2013. "Perspectives to Performance of Environment and Health Assessments and Models—From Outputs to Outcomes?," IJERPH, MDPI, vol. 10(7), pages 1-22, June.
    15. Peterson, Christina & Skolits, Gary, 2020. "Value for money: A utilization-focused approach to extending the foundation and contribution of economic evaluation," Evaluation and Program Planning, Elsevier, vol. 80(C).
    16. Sharma, Bhanu & Robinson, Jackie & Arhen, Benjamin B. & Timmons, Brian W. & Heal, Bryan & Warner, Marika, 2025. "Evaluating sport-for-development outcome measures used in a living lab setting: Process, improvements, and insights," Evaluation and Program Planning, Elsevier, vol. 112(C).
    17. Sobelson, Robyn K. & Young, Andrea C., 2013. "Evaluation of a federally funded workforce development program: The Centers for Public Health Preparedness," Evaluation and Program Planning, Elsevier, vol. 37(C), pages 50-57.
    18. Kalpazidou Schmidt, Evanthia & Graversen, Ebbe Krogh, 2020. "Developing a conceptual evaluation framework for gender equality interventions in research and innovation," Evaluation and Program Planning, Elsevier, vol. 79(C).
    19. Hudon, Catherine & Chouinard, Maud-Christine & Brousselle, Astrid & Bisson, Mathieu & Danish, Alya, 2020. "Evaluating complex interventions in real context: Logic analysis of a case management program for frequent users of healthcare services," Evaluation and Program Planning, Elsevier, vol. 79(C).
    20. Jabeen, Sumera, 2018. "Unintended outcomes evaluation approach: A plausible way to evaluate unintended outcomes of social development programmes," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 262-274.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:55:y:2016:i:c:p:103-111. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.