IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v95y2022ics0149718922000477.html

Leveraging integrated data for program evaluation: Recommendations from the field

Author

Listed:
  • Zanti, Sharon
  • Berkowitz, Emily
  • Katz, Matthew
  • Nelson, Amy Hawn
  • Burnett, T.C.
  • Culhane, Dennis
  • Zhou, Yixi

Abstract

Use of administrative data to inform decision making is now commonplace throughout the public sector, including program and policy evaluation. While reuse of these data can reduce costs, improve methodologies, and shorten timelines, challenges remain. This article informs evaluators about the growing field of Integrated Data Systems (IDS), and how to leverage cross-sector administrative data in evaluation work. This article is informed by three sources: a survey of current data integration efforts in the United States (U.S.) (N=63), informational interviews with experts, and internal knowledge cultivated through Actionable Intelligence for Social Policy’s (AISP) 12+ years of work in the field. A brief discussion of the U.S. data integration context and history is provided, followed by discussion of tangible recommendations for evaluators, examples of evaluations relying on integrated data, and a list of U.S. IDS sites with publicly available processes for external data requests. Despite the challenges associated with reusing administrative data for program evaluation, IDS offer evaluators a new set of tools for leveraging data across institutional silos.

Suggested Citation

  • Zanti, Sharon & Berkowitz, Emily & Katz, Matthew & Nelson, Amy Hawn & Burnett, T.C. & Culhane, Dennis & Zhou, Yixi, 2022. "Leveraging integrated data for program evaluation: Recommendations from the field," Evaluation and Program Planning, Elsevier, vol. 95(C).
  • Handle: RePEc:eee:epplan:v:95:y:2022:i:c:s0149718922000477
    DOI: 10.1016/j.evalprogplan.2022.102093
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718922000477
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.evalprogplan.2022.102093?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Aneesh M.R., 2021. "Quality of Drinking Water and Sanitation in India," Indian Journal of Human Development, , vol. 15(1), pages 138-152, April.
    2. Murphy, Kelly & Moore, Kristin Anderson & Redd, Zakia & Malm, Karin, 2017. "Trauma-informed child welfare systems and children's well-being: A longitudinal evaluation of KVC's bridging the way home initiative," Children and Youth Services Review, Elsevier, vol. 75(C), pages 23-34.
    3. Daniel T. L. Shek, 2021. "COVID-19 and Quality of Life: Twelve Reflections," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 16(1), pages 1-11, February.
    4. Krista J. Li, 2021. "Behavior-Based Quality Discrimination," Manufacturing & Service Operations Management, INFORMS, vol. 23(2), pages 425-436, March.
    5. ., 2021. "Quality Jobs Fund," Chapters, in: Strategic Community Partnerships, Philanthropy, and Nongovernmental Organization, chapter 9, pages 59-69, Edward Elgar Publishing.
    6. Putnam-Hornstein, E. & Cleves, M.A. & Licht, R. & Needell, B., 2013. "Risk of fatal injury in young children following abuse allegations: Evidence from a prospective, population-based study," American Journal of Public Health, American Public Health Association, vol. 103(10), pages 39-44.
    7. Raj Chetty & Nathaniel Hendren & Patrick Kline & Emmanuel Saez, 2014. "Where is the land of Opportunity? The Geography of Intergenerational Mobility in the United States," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1553-1623.
    8. Li, Xiang & Qi, Xiangtong, 2021. "On pricing and quality decisions with risk aversion," Omega, Elsevier, vol. 98(C).
    9. Robert M. Goerge, 2018. "Barriers to Accessing State Data and Approaches to Addressing Them," The ANNALS of the American Academy of Political and Social Science, , vol. 675(1), pages 122-137, January.
    10. ., 2021. "Qualitative methods training," Chapters, in: How to Enhance Your Research, chapter 32, pages 105-107, Edward Elgar Publishing.
    11. Green, Beth L. & Ayoub, Catherine & Bartlett, Jessica Dym & Furrer, Carrie & Von Ende, Adam & Chazan-Cohen, Rachel & Klevens, Joanne & Nygren, Peggy, 2015. "It's not as simple as it sounds: Problems and solutions in accessing and using administrative child welfare data for evaluating the impact of early childhood interventions," Children and Youth Services Review, Elsevier, vol. 57(C), pages 40-49.
    12. David S. Johnson & Catherine Massey & Amy O’Hara, 2015. "The Opportunities and Challenges of Using Administrative Data Linkages to Evaluate Mobility," The ANNALS of the American Academy of Political and Social Science, , vol. 657(1), pages 247-264, January.
    13. Barnow, Burt S., 1986. "Evaluating employment and training programs," Evaluation and Program Planning, Elsevier, vol. 9(1), pages 63-72, January.
    14. Evans, Elizabeth & Anglin, M. Douglas & Urada, Darren & Yang, Joy, 2011. "Promising practices for delivery of court-supervised substance abuse treatment: Perspectives from six high-performing California counties operating Proposition 36," Evaluation and Program Planning, Elsevier, vol. 34(2), pages 124-134, May.
    15. Oh, Sehun & DiNitto, Diana M. & Powers, Daniel A., 2020. "A longitudinal evaluation of government-sponsored job skills training and basic employment services among U.S. baby boomers with economic disadvantages," Evaluation and Program Planning, Elsevier, vol. 82(C).
    16. Lemieux, Ashley J. & Roy, Laurence & Martin, Michael S. & Latimer, Eric A. & Crocker, Anne G., 2017. "Justice involvement among homeless individuals with mental illnesses: Are self-report and administrative measures comparable?," Evaluation and Program Planning, Elsevier, vol. 61(C), pages 86-95.
    17. Kim, Hyunil & Jonson-Reid, Melissa & Kohl, Patricia & Chiang, Chien-jen & Drake, Brett & Brown, Derek & McBride, Tim & Guo, Shenyang, 2020. "Latent class analysis risk profiles: An effective method to predict a first re-report of maltreatment?," Evaluation and Program Planning, Elsevier, vol. 80(C).
    18. Harrison Ng Chok & Judy Mannix & Cathy Dickson & Lesley Wilkes, 2019. "Shining a light on refugee nurses," Journal of Clinical Nursing, John Wiley & Sons, vol. 28(9-10), pages 1363-1364, May.
    19. Lundgren, Lena M. & Schilling, Robert F. & Ferguson, Faith & Davis, Karen & Amodeo, Maryann, 2003. "Examining drug treatment program entry of injection drug users: human capital and institutional disaffiliation," Evaluation and Program Planning, Elsevier, vol. 26(2), pages 123-132, May.
    20. , Nabila, 2021. "Improve The Quality Of Human Resources In Indonesia," OSF Preprints tpkhv, Center for Open Science.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Blackburn, Natalie A. & Philbrick, Sarah & Patel, Sheila V. & Cance, Jessica D. & Elek, Elvira & Gibbons, Brent & Khan, Sazid & Lambdin, Barrot H. & Graham, Phillip W., 2025. "Advancing methods for program evaluation in substance use by strengthening the application of epidemiology, economic evaluation, and implementation science: Reflections on the opportunities for synergy," Evaluation and Program Planning, Elsevier, vol. 111(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Grimaldi, Michele & Coppola, Francesca & Fasolino, Isidoro, 2023. "A crime risk-based approach for urban planning. A methodological proposal," Land Use Policy, Elsevier, vol. 126(C).
    2. Carrilho, João & Dgedge, Gustavo & Santos, Pedro Manuel Pinto dos & Trindade, Jorge, 2024. "Sustainable land use: Policy implications of systematic land regularization in Mozambique," Land Use Policy, Elsevier, vol. 138(C).
    3. Timothy M. Smeeding, 2018. "The PSID in Research and Policy," The ANNALS of the American Academy of Political and Social Science, , vol. 680(1), pages 29-47, November.
    4. Cobb-Clark, Deborah A. & Dahmann, Sarah C. & Salamanca, Nicolás & Zhu, Anna, 2022. "Intergenerational disadvantage: Learning about equal opportunity from social assistance receipt," Labour Economics, Elsevier, vol. 79(C).
    5. David B. Grusky & Timothy M. Smeeding & C. Matthew Snipp, 2015. "A New Infrastructure for Monitoring Social Mobility in the United States," The ANNALS of the American Academy of Political and Social Science, , vol. 657(1), pages 63-82, January.
    6. Diogo G. C. Britto & Alexandre Fonseca & Paolo Pinotti & Breno Sampaio & Lucas Warwar, 2022. "Intergenerational Mobility in the Land of Inequality," CESifo Working Paper Series 10004, CESifo.
    7. Boneva, Teodora & Golin, Marta & Kaufmann, Katja Maria & Rauh, Christopher, 2022. "Beliefs about Maternal Labor Supply," IZA Discussion Papers 15788, IZA Network @ LISER.
    8. Jo Blanden, 2019. "Intergenerational income persistence," World of Labour, LISER, pages 176-176, January.
    9. Maia Güell & Michele Pellizzari & Giovanni Pica & José V. Rodríguez Mora, 2018. "Correlating Social Mobility and Economic Outcomes," Economic Journal, Royal Economic Society, vol. 128(612), pages 353-403, July.
    10. Thiago Henrique Leite & Marina Silva Cunha, 2024. "Intergenerational Educational Mobility in Brazil: Trends and Patterns," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 174(1), pages 19-45, August.
    11. Uguccioni, James, 2022. "The long-run effects of parental unemployment in childhood," CLEF Working Paper Series 45, Canadian Labour Economics Forum (CLEF), University of Waterloo.
    12. Julia Philipp, 2023. "Gendered university major choice: the role of intergenerational transmission," Journal of Population Economics, Springer;European Society for Population Economics, vol. 36(2), pages 1049-1097, April.
    13. repec:spo:wpmain:info:hdl:2441/6s39gt704s95upu27ma7s3p6q8 is not listed on IDEAS
    14. M. Shahe Emran & Francisco H. G. Ferreira & Yajing Jiang & Yan Sun, 2023. "Occupational dualism and intergenerational educational mobility in the rural economy: evidence from China and India," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 21(3), pages 743-773, September.
    15. Mattéo Godin & Jean Hindriks, 2018. "An international comparison of school systems based on social mobility," Economie et Statistique / Economics and Statistics, Institut National de la Statistique et des Etudes Economiques (INSEE), issue 499, pages 61-78.
    16. Pablo A. Mitnik & Erin Cumberworth & David B. Grusky, 2016. "Social Mobility in a High-Inequality Regime," The ANNALS of the American Academy of Political and Social Science, , vol. 663(1), pages 140-184, January.
    17. Simon Halphen Boserup & Wojciech Kopczuk & Claus Thustrup Kreiner, 2018. "Born with a Silver Spoon? Danish Evidence on Wealth Inequality in Childhood," Economic Journal, Royal Economic Society, vol. 128(612), pages 514-544, July.
    18. Nan Zhao & Wanqing Liao & Jun Xia & Zizhe Zhang, 2023. "The effect of intergenerational mobility on family education investment: evidence from China," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    19. Bernt Bratsberg & Simen Markussen & Oddbjørn Raaum & Knut Røed & Ole Røgeberg, 2023. "Trends in Assortative Mating and Offspring Outcomes," The Economic Journal, Royal Economic Society, vol. 133(651), pages 928-950.
    20. Delis, Manthos & Iosifidi, Maria & Pnevmatikos, Lampros & Tsiritakis, Emmanuel, 2023. "Credit negotiations," MPRA Paper 123003, University Library of Munich, Germany, revised 12 Dec 2024.
    21. Ling Zhao & Anquan Zou & Minghua Xiong & Jun Liu, 2023. "Supply Chain Performance with a Downside-Risk-Averse Retailer and Strategic Customers," Sustainability, MDPI, vol. 15(4), pages 1-18, February.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:95:y:2022:i:c:s0149718922000477. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.