IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v95y2022ics0149718922000477.html
   My bibliography  Save this article

Leveraging integrated data for program evaluation: Recommendations from the field

Author

Listed:
  • Zanti, Sharon
  • Berkowitz, Emily
  • Katz, Matthew
  • Nelson, Amy Hawn
  • Burnett, T.C.
  • Culhane, Dennis
  • Zhou, Yixi

Abstract

Use of administrative data to inform decision making is now commonplace throughout the public sector, including program and policy evaluation. While reuse of these data can reduce costs, improve methodologies, and shorten timelines, challenges remain. This article informs evaluators about the growing field of Integrated Data Systems (IDS), and how to leverage cross-sector administrative data in evaluation work. This article is informed by three sources: a survey of current data integration efforts in the United States (U.S.) (N=63), informational interviews with experts, and internal knowledge cultivated through Actionable Intelligence for Social Policy’s (AISP) 12+ years of work in the field. A brief discussion of the U.S. data integration context and history is provided, followed by discussion of tangible recommendations for evaluators, examples of evaluations relying on integrated data, and a list of U.S. IDS sites with publicly available processes for external data requests. Despite the challenges associated with reusing administrative data for program evaluation, IDS offer evaluators a new set of tools for leveraging data across institutional silos.

Suggested Citation

  • Zanti, Sharon & Berkowitz, Emily & Katz, Matthew & Nelson, Amy Hawn & Burnett, T.C. & Culhane, Dennis & Zhou, Yixi, 2022. "Leveraging integrated data for program evaluation: Recommendations from the field," Evaluation and Program Planning, Elsevier, vol. 95(C).
  • Handle: RePEc:eee:epplan:v:95:y:2022:i:c:s0149718922000477
    DOI: 10.1016/j.evalprogplan.2022.102093
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718922000477
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.evalprogplan.2022.102093?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Murphy, Kelly & Moore, Kristin Anderson & Redd, Zakia & Malm, Karin, 2017. "Trauma-informed child welfare systems and children's well-being: A longitudinal evaluation of KVC's bridging the way home initiative," Children and Youth Services Review, Elsevier, vol. 75(C), pages 23-34.
    2. Raj Chetty & Nathaniel Hendren & Patrick Kline & Emmanuel Saez, 2014. "Where is the land of Opportunity? The Geography of Intergenerational Mobility in the United States," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1553-1623.
    3. Evans, Elizabeth & Anglin, M. Douglas & Urada, Darren & Yang, Joy, 2011. "Promising practices for delivery of court-supervised substance abuse treatment: Perspectives from six high-performing California counties operating Proposition 36," Evaluation and Program Planning, Elsevier, vol. 34(2), pages 124-134, May.
    4. Lemieux, Ashley J. & Roy, Laurence & Martin, Michael S. & Latimer, Eric A. & Crocker, Anne G., 2017. "Justice involvement among homeless individuals with mental illnesses: Are self-report and administrative measures comparable?," Evaluation and Program Planning, Elsevier, vol. 61(C), pages 86-95.
    5. Li, Xiang & Qi, Xiangtong, 2021. "On pricing and quality decisions with risk aversion," Omega, Elsevier, vol. 98(C).
    6. Krista J. Li, 2021. "Behavior-Based Quality Discrimination," Manufacturing & Service Operations Management, INFORMS, vol. 23(2), pages 425-436, March.
    7. Putnam-Hornstein, E. & Cleves, M.A. & Licht, R. & Needell, B., 2013. "Risk of fatal injury in young children following abuse allegations: Evidence from a prospective, population-based study," American Journal of Public Health, American Public Health Association, vol. 103(10), pages 39-44.
    8. Robert M. Goerge, 2018. "Barriers to Accessing State Data and Approaches to Addressing Them," The ANNALS of the American Academy of Political and Social Science, , vol. 675(1), pages 122-137, January.
    9. Green, Beth L. & Ayoub, Catherine & Bartlett, Jessica Dym & Furrer, Carrie & Von Ende, Adam & Chazan-Cohen, Rachel & Klevens, Joanne & Nygren, Peggy, 2015. "It's not as simple as it sounds: Problems and solutions in accessing and using administrative child welfare data for evaluating the impact of early childhood interventions," Children and Youth Services Review, Elsevier, vol. 57(C), pages 40-49.
    10. Lundgren, Lena M. & Schilling, Robert F. & Ferguson, Faith & Davis, Karen & Amodeo, Maryann, 2003. "Examining drug treatment program entry of injection drug users: human capital and institutional disaffiliation," Evaluation and Program Planning, Elsevier, vol. 26(2), pages 123-132, May.
    11. Aneesh M.R., 2021. "Quality of Drinking Water and Sanitation in India," Indian Journal of Human Development, , vol. 15(1), pages 138-152, April.
    12. David S. Johnson & Catherine Massey & Amy O’Hara, 2015. "The Opportunities and Challenges of Using Administrative Data Linkages to Evaluate Mobility," The ANNALS of the American Academy of Political and Social Science, , vol. 657(1), pages 247-264, January.
    13. Oh, Sehun & DiNitto, Diana M. & Powers, Daniel A., 2020. "A longitudinal evaluation of government-sponsored job skills training and basic employment services among U.S. baby boomers with economic disadvantages," Evaluation and Program Planning, Elsevier, vol. 82(C).
    14. Harrison Ng Chok & Judy Mannix & Cathy Dickson & Lesley Wilkes, 2019. "Shining a light on refugee nurses," Journal of Clinical Nursing, John Wiley & Sons, vol. 28(9-10), pages 1363-1364, May.
    15. ., 2021. "Quality Jobs Fund," Chapters, in: Strategic Community Partnerships, Philanthropy, and Nongovernmental Organization, chapter 9, pages 59-69, Edward Elgar Publishing.
    16. ., 2021. "Qualitative methods training," Chapters, in: How to Enhance Your Research, chapter 32, pages 105-107, Edward Elgar Publishing.
    17. Barnow, Burt S., 1986. "Evaluating employment and training programs," Evaluation and Program Planning, Elsevier, vol. 9(1), pages 63-72, January.
    18. Kim, Hyunil & Jonson-Reid, Melissa & Kohl, Patricia & Chiang, Chien-jen & Drake, Brett & Brown, Derek & McBride, Tim & Guo, Shenyang, 2020. "Latent class analysis risk profiles: An effective method to predict a first re-report of maltreatment?," Evaluation and Program Planning, Elsevier, vol. 80(C).
    19. Daniel T. L. Shek, 2021. "COVID-19 and Quality of Life: Twelve Reflections," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 16(1), pages 1-11, February.
    20. , Nabila, 2021. "Improve The Quality Of Human Resources In Indonesia," OSF Preprints tpkhv, Center for Open Science.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Grimaldi, Michele & Coppola, Francesca & Fasolino, Isidoro, 2023. "A crime risk-based approach for urban planning. A methodological proposal," Land Use Policy, Elsevier, vol. 126(C).
    2. Cobb-Clark, Deborah A. & Dahmann, Sarah C. & Salamanca, Nicolás & Zhu, Anna, 2022. "Intergenerational disadvantage: Learning about equal opportunity from social assistance receipt," Labour Economics, Elsevier, vol. 79(C).
    3. Timothy M. Smeeding, 2018. "The PSID in Research and Policy," The ANNALS of the American Academy of Political and Social Science, , vol. 680(1), pages 29-47, November.
    4. David B. Grusky & Timothy M. Smeeding & C. Matthew Snipp, 2015. "A New Infrastructure for Monitoring Social Mobility in the United States," The ANNALS of the American Academy of Political and Social Science, , vol. 657(1), pages 63-82, January.
    5. Jo Blanden, 2019. "Intergenerational income persistence," IZA World of Labor, Institute of Labor Economics (IZA), pages 176-176, January.
    6. Maia Güell & Michele Pellizzari & Giovanni Pica & José V. Rodríguez Mora, 2018. "Correlating Social Mobility and Economic Outcomes," Economic Journal, Royal Economic Society, vol. 128(612), pages 353-403, July.
    7. Andrés Rodríguez-Pose & Michael Storper, 2020. "Housing, urban growth and inequalities: The limits to deregulation and upzoning in reducing economic and spatial inequality," Urban Studies, Urban Studies Journal Limited, vol. 57(2), pages 223-248, February.
    8. Yann Algan & Elizabeth Beasley & Frank Vitaro & Richard Tremblay, 2014. "The Impact of Non-Cognitive Skills Training on Academic and Non-academic Trajectories: From Childhood to Early Adulthood," Working Papers hal-03429906, HAL.
    9. M. Shahe Emran & Francisco H. G. Ferreira & Yajing Jiang & Yan Sun, 2023. "Occupational dualism and intergenerational educational mobility in the rural economy: evidence from China and India," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 21(3), pages 743-773, September.
    10. Ling Zhao & Anquan Zou & Minghua Xiong & Jun Liu, 2023. "Supply Chain Performance with a Downside-Risk-Averse Retailer and Strategic Customers," Sustainability, MDPI, vol. 15(4), pages 1-18, February.
    11. Andreas Fagereng & Luigi Guiso & Davide Malacrino & Luigi Pistaferri, 2020. "Heterogeneity and Persistence in Returns to Wealth," Econometrica, Econometric Society, vol. 88(1), pages 115-170, January.
    12. Adrian Adermon & Mikael Lindahl & Daniel Waldenström, 2018. "Intergenerational Wealth Mobility and the Role of Inheritance: Evidence from Multiple Generations," Economic Journal, Royal Economic Society, vol. 128(612), pages 482-513, July.
    13. Francesco Andreoli & Eugenio Peluso, 2016. "So close yet so unequal: Reconsidering spatial inequality in U.S. cities," Working Papers 21/2016, University of Verona, Department of Economics.
    14. Martti Kaila & Emily Nix & Krista Riukula, 2021. "Disparate Impacts of Job Loss by Parental Income and Implications for Intergenerational Mobility," Opportunity and Inclusive Growth Institute Working Papers 53, Federal Reserve Bank of Minneapolis.
    15. Bratu, Cristina & Bolotnyy, Valentin, 2023. "Immigrant intergenerational mobility: A focus on childhood environment," European Economic Review, Elsevier, vol. 151(C).
    16. Simon Halphen Boserup & Wojciech Kopczuk & Claus Thustrup Kreiner, 2018. "Born with a Silver Spoon? Danish Evidence on Wealth Inequality in Childhood," Economic Journal, Royal Economic Society, vol. 128(612), pages 514-544, July.
    17. Donald R. Davis & Jonathan I. Dingel & Joan Monras & Eduardo Morales, 2019. "How Segregated Is Urban Consumption?," Journal of Political Economy, University of Chicago Press, vol. 127(4), pages 1684-1738.
    18. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.
    19. Naguib, Costanza, 2019. "Estimating the Heterogeneous Impact of the Free Movement of Persons on Relative Wage Mobility," Economics Working Paper Series 1903, University of St. Gallen, School of Economics and Political Science.
    20. Bachas, Pierre & Fattal Jaef, Roberto N. & Jensen, Anders, 2019. "Size-dependent tax enforcement and compliance: Global evidence and aggregate implications," Journal of Development Economics, Elsevier, vol. 140(C), pages 203-222.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:95:y:2022:i:c:s0149718922000477. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.