IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v33y2009i3p281-306.html

Old Wine in New Skins: The Sensitivity of Established Findings to New Methods

Author

Listed:
  • E. Michael Foster

    (University of North Carolina at Chapel Hill, emfoster@unc.edu)

  • Elizabeth Wiley-Exley

    (University of North Carolina at Chapel Hill)

  • Leonard Bickman

    (Vanderbilt University)

Abstract

Findings from an evaluation of a model system for delivering mental health services to youth were reassessed to determine the robustness of key findings to the use of methodologies unavailable to the original analysts. These analyses address a key concern about earlier findings—that the quasi-experimental design involved the comparison of two noncomparable groups. The authors employed propensity score methodology to reconsider between-group baseline differences in observed characteristics of participating families. The authors also considered the possible effect of unobserved between-group differences. The data support previous studies that show few differences in outcomes, but the findings are sensitive to unobserved heterogeneity.

Suggested Citation

  • E. Michael Foster & Elizabeth Wiley-Exley & Leonard Bickman, 2009. "Old Wine in New Skins: The Sensitivity of Established Findings to New Methods," Evaluation Review, , vol. 33(3), pages 281-306, June.
    • Handle: RePEc:sae:evarev:v:33:y:2009:i:3:p:281-306
    DOI: 10.1177/0193841X09334028
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X09334028
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X09334028?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    2. Verbeek, Marno & Nijman, Theo, 1992. "Testing for Selectivity Bias in Panel Data Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 33(3), pages 681-703, August.
    3. Lee, Myoung-jae, 2005. "Micro-Econometrics for Policy, Program and Treatment Effects," OUP Catalogue, Oxford University Press, number 9780199267699.
    4. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70, Elsevier.
    5. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    6. Nijman, T.E. & Verbeek, M.J.C.M., 1992. "Testing for selectivity in panel data models," Other publications TiSEM 7ec34a6c-1d84-4052-971c-d, Tilburg University, School of Economics and Management.
    7. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. E. Santarelli & H. T. Tran, 2013. "Diversification Strategies and Firm Performance: A Sample Selection Approach," Working Papers wp896, Dipartimento Scienze Economiche, Universita' di Bologna.
    3. Foster, E. Michael & McCombs-Thornton, Kimberly, 2013. "Child welfare and the challenge of causal inference," Children and Youth Services Review, Elsevier, vol. 35(7), pages 1130-1142.
    4. Nicoletti, Cheti, 2006. "Nonresponse in dynamic panel data models," Journal of Econometrics, Elsevier, vol. 132(2), pages 461-489, June.
    5. Jiang, Miao & Foster, E. Michael & Gibson-Davis, Christina M., 2010. "The effect of WIC on breastfeeding: A new look at an established relationship," Children and Youth Services Review, Elsevier, vol. 32(2), pages 264-273, February.
    6. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    7. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    8. Jan Fałkowski & Maciej Jakubowski & Paweł Strawiński, 2014. "Returns from income strategies in rural Poland," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 22(1), pages 139-178, January.
    9. Haiyang Lu & Peng Nie & Alfonso Sousa-Poza, 2021. "The Effect of Parental Educational Expectations on Adolescent Subjective Well-Being and the Moderating Role of Perceived Academic Pressure: Longitudinal Evidence for China," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 14(1), pages 117-137, February.
    10. Christian Volpe Martincus & Jerónimo Carballo, 2010. "Is Export Promotion Effective in Developing Countries? Firm-Level Evidence on the Intensive and Extensive Margins of Exports," IDB Publications (Working Papers) 36763, Inter-American Development Bank.
    11. repec:dau:papers:123456789/1637 is not listed on IDEAS
    12. Lionel WILNER, 2019. "The Dynamics of Individual Happiness," Working Papers 2019-18, Center for Research in Economics and Statistics.
    13. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    14. Patricia Cubí‐Mollá & Mireia Jofre‐Bonet & Victoria Serra‐Sastre, 2017. "Adaptation to health states: Sick yet better off?," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1826-1843, December.
    15. Anand, Paul & Gray, Alastair & Liberini, Federica & Roope, Laurence & Smith, Ron & Thomas, Ranjeeta, 2015. "Wellbeing over 50," The Journal of the Economics of Ageing, Elsevier, vol. 6(C), pages 68-78.
    16. Shabbar Jaffry & Yaseen Ghulam & Vyoma Shah, 2007. "Returns to Education in Pakistan," The Pakistan Development Review, Pakistan Institute of Development Economics, vol. 46(4), pages 833-852.
    17. Strijbosch, L.W.G. & Heuts, R.M.J., 1994. "Investigating several alternatives for estimating the lead time demand distribution in a continuous review inventory model," Other publications TiSEM dc1f886c-0122-4da9-9598-8, Tilburg University, School of Economics and Management.
    18. Paweł Strawiński, 2013. "Controlling for overlap in matching," Working Papers 2013-10, Faculty of Economic Sciences, University of Warsaw.
    19. Bubonya, Melisa & Cobb-Clark, Deborah A. & Wooden, Mark, 2017. "Mental health and productivity at work: Does what you do matter?," Labour Economics, Elsevier, vol. 46(C), pages 150-165.
    20. Doreen Wing Han Au & Thomas F. Crossley & Martin Schellhorn, 2005. "The effect of health changes and long‐term health on the work activity of older Canadians," Health Economics, John Wiley & Sons, Ltd., vol. 14(10), pages 999-1018, October.
    21. E. Michael Foster & Grace Y. Fang, 2004. "Alternative Methods for Handling Attrition," Evaluation Review, , vol. 28(5), pages 434-464, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:33:y:2009:i:3:p:281-306. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.