IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v47y2023i1p104-122.html
   My bibliography  Save this article

Insights From the Health Profession Opportunity Grant Program’s Three-Armed, Multi-Site Experiment for Policy Learning and Evaluation Practice

Author

Listed:
  • Laura R. Peck
  • Hilary Bruck
  • Nicole Constance

Abstract

Administered by the Office of Family Assistance in the U.S. Department of Health and Human Services’ Administration for Children and Families (ACF), the Health Profession Opportunity Grants (HPOG) Program provided education and training to Temporary Assistance for Needy Families (TANF) recipients and other adults with low incomes for occupations in the healthcare field. The impact evaluation for the first cohort of grantees (HPOG 1.0) leveraged the program’s implementation across many locations, using a three-armed evaluation design (including a second treatment arm) in some places, as a way to examine whether any of three selected program components, or enhancements, contributed to the program’s overall impact. This article tells the story of the evaluation and draws lessons from that experience, discussing implications for future implementation of multi-armed experiments in a multi-site evaluation.

Suggested Citation

  • Laura R. Peck & Hilary Bruck & Nicole Constance, 2023. "Insights From the Health Profession Opportunity Grant Program’s Three-Armed, Multi-Site Experiment for Policy Learning and Evaluation Practice," Evaluation Review, , vol. 47(1), pages 104-122, February.
    • Handle: RePEc:sae:evarev:v:47:y:2023:i:1:p:104-122
    DOI: 10.1177/0193841X221133420
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X221133420
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X221133420?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Howard S. Bloom & Carolyn J. Hill & James A. Riccio, 2003. "Linking program implementation and effectiveness: Lessons from a pooled sample of welfare-to-work experiments," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 22(4), pages 551-575.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Geoffrey L. Wallace & Robert Haveman, 2007. "The implications of differences between employer and worker employment|earnings reports for policy evaluation," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(4), pages 737-754.
    2. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.
    3. Helen Lee & Sarah Shea Crowne & Melanie Estarziau & Keith Kranker & Charles Michalopoulos & Anne Warren & Tod Mijanovich & Jill H. Filene & Anne Duggan & Virginia Knox, "undated". "The Effects of Home Visiting on Prenatal Health, Birth Outcomes, and Health Care Use in the First Year of Life: Final Implementation and Impact Findings from the Mother and Infant Home Visiting Progra," Mathematica Policy Research Reports a9626a8d90bf4f01811d0c9d7, Mathematica Policy Research.
    4. Stefanie Behncke & Markus Frölich & Michael Lechner, 2010. "Unemployed and their caseworkers: should they be friends or foes?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(1), pages 67-92, January.
    5. repec:mpr:mprres:7520 is not listed on IDEAS
    6. Adam M. Butz, 2015. "Administrative Privatization and Employment Outcomes in the Implementation of Temporary Assistance to Needy Families," Evaluation Review, , vol. 39(4), pages 363-394, August.
    7. Andrew McEachin & Thurston Domina & Andrew Penner, 2020. "Heterogeneous Effects of Early Algebra across California Middle Schools," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(3), pages 772-800, June.
    8. Stephen Bell & Daniel Gubits & David Stapleton & David Wittenburg & Michelle Derr & Arkadipta Ghosh & Sara Ansell & David Greenberg, 2011. "BOND Implementation and Evaluation: Evaluation Analysis Plan," Mathematica Policy Research Reports 795952932ec747588c128efa0, Mathematica Policy Research.
    9. Sarah Croake & Priyanka Anand & Christopher Jones & Katherine Morrison & Cara Orfield & David Stapleton & Denise Hoffman & David R. Mann & Judy Geyer & Daniel Gubits & Stephen Bell & Andrew McGuirk & , "undated". "BOND Implementation and Evaluation: 2017 Stage 1 Interim Process, Participation, and Impact Report," Mathematica Policy Research Reports b00eb83020fb42e185d69f979, Mathematica Policy Research.
    10. Charles Michalopoulos & Kristen Faucetta & Carolyn J. Hill & Zimena A. Portilla & Lori Burrell & Helen Lee & Anne Duggan & Virginia Knox, "undated". "Impacts on Family Outcomes of Evidence-Based Early Childhood Home Visiting: Results from the Mother and Infant Home Visiting Program Evaluation," Mathematica Policy Research Reports 3adcbd3368c545679a6784b8a, Mathematica Policy Research.
    11. Virginia Knox & Carolyn J. Hill & Gordon Berlin, 2018. "Can Evidence-Based Policy Ameliorate the Nation’s Social Problems?," The ANNALS of the American Academy of Political and Social Science, , vol. 678(1), pages 166-179, July.
    12. David Greenberg & Burt S. Barnow, 2014. "Flaws in Evaluations of Social Programs," Evaluation Review, , vol. 38(5), pages 359-387, October.
    13. Jon Baron, 2018. "A Brief History of Evidence-Based Policy," The ANNALS of the American Academy of Political and Social Science, , vol. 678(1), pages 40-50, July.
    14. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    15. Peter Saunders & Judith Brown, 2004. "Labour Market Conditions, Applications and Grants of disability support Pension (DSP) in Australia," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 7(3), pages 395-410, September.
    16. Oscar Mitnik, 2008. "How do Training Programs Assign Participants to Training? Characterizing the Assignment Rules of Government Agencies for Welfare-to-Work Programs in California," Working Papers 0907, University of Miami, Department of Economics.
    17. Jochen Michaelis & Alexander Spermann, 2009. "Geringqualifizierte Arbeit, Marktlöhne und Sozialpolitik: Konzepte für Deutschland," MAGKS Papers on Economics 200920, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    18. David Stapleton & David Wittenburg & Daniel Gubits & David Judkins & David R. Mann & Andrew McGuirk, "undated". "BOND Implementation and Evaluation: First-Year Snapshot of Earnings and Benefit Impacts for Stage 1," Mathematica Policy Research Reports ef02b16048974780b148458f9, Mathematica Policy Research.
    19. Judy Geyer & Daniel Gubits & Stephen Bell & Tyler Morrill & Denise Hoffman & Sarah Croake & Katie Morrison & David Judkins & David Stapleton, "undated". "BOND Implementation and Evaluation: 2017 Stage 2 Interim Process, Participation, and Impact Report," Mathematica Policy Research Reports 3200a977680046ed8df3b0e65, Mathematica Policy Research.
    20. Margaretha Buurman & Robert Dur, 2012. "Incentives and the Sorting of Altruistic Agents into Street-Level Bureaucracies," Scandinavian Journal of Economics, Wiley Blackwell, vol. 114(4), pages 1318-1345, December.
    21. Ankita Patnaik & Michael Levere & Gina Livermore & Arif Mamun & Jeffrey Hemmeter, 2021. "Promoting Readiness of Minors in Supplemental Security Income (PROMISE): Early Impacts from a Multi-Site Random Assignment Evaluation," Evaluation Review, , vol. 45(5), pages 228-270, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:47:y:2023:i:1:p:104-122. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.