IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v58y2016icp88-97.html
   My bibliography  Save this article

Steps towards incorporating heterogeneities into program theory: A case study of a data-driven approach

Author

Listed:
  • Sridharan, Sanjeev
  • Jones, Bobby
  • Caudill, Barry
  • Nakaima, April

Abstract

This paper describes a framework that can help refine program theory through data explorations and stakeholder dialogue. The framework incorporates the following steps: a recognition that program implementation might need to be multi-phased for a number of interventions, the need to take stock of program theory, the application of pattern recognition methods to help identify heterogeneous program mechanisms, and stakeholder dialogue to refine the program. As part of the data exploration, a method known as developmental trajectories is implemented to learn about heterogeneous trajectories of outcomes in longitudinal evaluations. This method identifies trajectory clusters and also can estimate different treatment impacts for the various groups. The framework is highlighted with data collected in an evaluation of an alcohol risk-reduction program delivered in a college fraternity setting. The framework discussed in the paper is informed by a realist focus on “what works for whom under what contexts.” The utility of the framework in contributing to a dialogue on heterogeneous mechanism and subsequent implementation is described. The connection of the ideas in paper to a ‘learning through principled discovery’ approach is also described.

Suggested Citation

  • Sridharan, Sanjeev & Jones, Bobby & Caudill, Barry & Nakaima, April, 2016. "Steps towards incorporating heterogeneities into program theory: A case study of a data-driven approach," Evaluation and Program Planning, Elsevier, vol. 58(C), pages 88-97.
    • Handle: RePEc:eee:epplan:v:58:y:2016:i:c:p:88-97
    DOI: 10.1016/j.evalprogplan.2016.05.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718916300039
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2016.05.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Luc Anselin & Sanjeev Sridharan & Susan Gholston, 2007. "Using Exploratory Spatial Data Analysis to Leverage Social Indicator Databases: The Discovery of Interesting Patterns," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 82(2), pages 287-309, June.
    2. Amelia M. Haviland & Bobby L. Jones & Daniel S. Nagin, 2011. "Group-based Trajectory Modeling Extended to Account for Nonrandom Participant Attrition," Sociological Methods & Research, , vol. 40(2), pages 367-390, May.
    3. Sterman, J.D., 2006. "Learning from evidence in a complex world," American Journal of Public Health, American Public Health Association, vol. 96(3), pages 505-514.
    4. Sridharan, Sanjeev & Nakaima, April, 2011. "Ten steps to making evaluation matter," Evaluation and Program Planning, Elsevier, vol. 34(2), pages 135-146, May.
    5. Amelia Haviland & Daniel Nagin, 2005. "Causal inferences with group based trajectory models," Psychometrika, Springer;The Psychometric Society, vol. 70(3), pages 557-578, September.
    6. Bobby L. Jones & Daniel S. Nagin, 2007. "Advances in Group-Based Trajectory Modeling and an SAS Procedure for Estimating Them," Sociological Methods & Research, , vol. 35(4), pages 542-571, May.
    7. Djebbari, Habiba & Smith, Jeffrey, 2008. "Heterogeneous impacts in PROGRESA," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 64-80, July.
    8. Bobby L. Jones & Daniel S. Nagin & Kathryn Roeder, 2001. "A SAS Procedure Based on Mixture Models for Estimating Developmental Trajectories," Sociological Methods & Research, , vol. 29(3), pages 374-393, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sridharan, Sanjeev & Nakaima, April, 2023. "Learning from experiences of evaluators implementing theory-driven evaluations in diverse settings: Building on the contributions of John Mayne," Evaluation and Program Planning, Elsevier, vol. 97(C).
    2. Nakaima, April & Sridharan, Sanjeev & Gibson, Rachael, 2023. "Towards an evolutionary approach to learning from assumptions: Lessons from the evaluation of Dancing with Parkinson’s," Evaluation and Program Planning, Elsevier, vol. 97(C).
    3. Sridharan, Sanjeev & Nakaima, April, 2020. "Valuing and embracing complexity: How an understanding of complex interventions needs to shape our evaluation capacities building initiatives," Evaluation and Program Planning, Elsevier, vol. 80(C).
    4. Myrta Kohler & Hanna Mayer & Jürg Kesselring & Susi Saxer, 2020. "Urinary incontinence in stroke survivors – Development of a programme theory," Journal of Clinical Nursing, John Wiley & Sons, vol. 29(15-16), pages 3089-3096, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dirlam, Jonathan & Zheng, Hui, 2017. "Job satisfaction developmental trajectories and health: A life course perspective," Social Science & Medicine, Elsevier, vol. 178(C), pages 95-103.
    2. Min Hua Jen & Ron Johnston & Kelvyn Jones & Richard Harris & Axel Gandy, 2010. "International Variations In Life Expectancy: A Spatio‐Temporal Analysis," Tijdschrift voor Economische en Sociale Geografie, Royal Dutch Geographical Society KNAG, vol. 101(1), pages 73-90, February.
    3. McCuish, Evan C. & Corrado, Raymond R. & Hart, Stephen D. & DeLisi, Matt, 2015. "The role of symptoms of psychopathy in persistent violence over the criminal career into full adulthood," Journal of Criminal Justice, Elsevier, vol. 43(4), pages 345-356.
    4. LEBIHAN, Laetitia & MAO TAKONGMO, Charles Olivier, 2018. "Mathematics Trajectories and Risk Factors During Childhood," MPRA Paper 88612, University Library of Munich, Germany.
    5. Corrado, Raymond R. & McCuish, Evan C. & Hart, Stephen D. & DeLisi, Matt, 2015. "The role of psychopathic traits and developmental risk factors on offending trajectories from early adolescence to adulthood: A prospective study of incarcerated youth," Journal of Criminal Justice, Elsevier, vol. 43(4), pages 357-368.
    6. Tiziano Arduini & Eleonora Patacchini & Edoardo Rainone, 2020. "Treatment Effects With Heterogeneous Externalities," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 826-838, October.
    7. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    8. Wenjing Luo & Zhi Qiu & Yurika Yokoyama & Shengyuan Zheng, 2022. "Decision-Making Mechanism of Joint Activities for the Elderly and Children in Integrated Welfare Facilities: A Discussion Based on “Motivation–Constraint” Interaction Model," IJERPH, MDPI, vol. 19(16), pages 1-23, August.
    9. Jiannan Lu & Peng Ding & Tirthankar Dasgupta, 2018. "Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 540-567, October.
    10. Callaway, Brantly & Li, Tong & Oka, Tatsushi, 2018. "Quantile treatment effects in difference in differences models under dependence restrictions and with only two time periods," Journal of Econometrics, Elsevier, vol. 206(2), pages 395-413.
    11. Kimberly M. Thompson & Radboud J. Duintjer Tebbens, 2006. "Retrospective Cost‐Effectiveness Analyses for Polio Vaccination in the United States," Risk Analysis, John Wiley & Sons, vol. 26(6), pages 1423-1440, December.
    12. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    13. Patrick Sturgis & Louise Sullivan, 2008. "Exploring social mobility with latent trajectory groups," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(1), pages 65-88, January.
    14. Sakos, Grayson & Cerulli, Giovanni & Garbero, Alessandra, 2021. "Beyond the ATE: Idiosyncratic Effect Estimation to Uncover Distributional Impacts Results from 17 Impact Evaluations," 2021 Annual Meeting, August 1-3, Austin, Texas 314017, Agricultural and Applied Economics Association.
    15. Bruno Crépon & Gerard J. van den Berg, 2016. "Active Labor Market Policies," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 521-546, October.
    16. Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2023. "Towards data-driven project design: Providing optimal treatment rules for development projects," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    17. Ragdad Cani Miranti, 2021. "Is regional poverty converging across Indonesian districts? A distribution dynamics and spatial econometric approach," Asia-Pacific Journal of Regional Science, Springer, vol. 5(3), pages 851-883, October.
    18. Clark, Alexander M., 2013. "What are the components of complex interventions in healthcare? Theorizing approaches to parts, powers and the whole intervention," Social Science & Medicine, Elsevier, vol. 93(C), pages 185-193.
    19. Michael J. Kottelenberg & Steven F. Lehrer, 2017. "Targeted or Universal Coverage? Assessing Heterogeneity in the Effects of Universal Child Care," Journal of Labor Economics, University of Chicago Press, vol. 35(3), pages 609-653.
    20. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:58:y:2016:i:c:p:88-97. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.