IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v34y2010i6p455-486.html
   My bibliography  Save this article

Examination of the Empirical Research Environment of Program Evaluation: Methodology and Application

Author

Listed:
  • Yonatan Eyal

    (Myers-JDC-Brookdale Institute, Jerusalem, Israel, eyal.yonatan@gmail.com)

Abstract

The study dealt with a research environment in which the treatment effect is heterogeneous, and in which individuals use their assessments of the treatment effect to decide whether or not to enroll in an intervention program. In this article, a new methodology is proposed for examining the validity of the specified research environment in a given intervention program and database. The methodology is based on direct examination of the assumptions underlying that research environment. Comparison of the commonly used verification method with the one proposed here revealed that these methods are complementary rather than substitutional.

Suggested Citation

  • Yonatan Eyal, 2010. "Examination of the Empirical Research Environment of Program Evaluation: Methodology and Application," Evaluation Review, , vol. 34(6), pages 455-486, December.
    • Handle: RePEc:sae:evarev:v:34:y:2010:i:6:p:455-486
    DOI: 10.1177/0193841X10394980
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X10394980
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X10394980?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    3. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
    4. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August.
    5. Heckman, James J. & Schmierer, Daniel & Urzua, Sergio, 2010. "Testing the correlated random coefficient model," Journal of Econometrics, Elsevier, vol. 158(2), pages 177-203, October.
    6. Alberto Abadie & Joshua Angrist & Guido Imbens, 2002. "Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings," Econometrica, Econometric Society, vol. 70(1), pages 91-117, January.
    7. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 487-535.
    8. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    9. Eyal, Yonatan & Beenstock, Michael, 2008. "Sign reversal in LIVE treatment effect estimates: The effect of vocational training on unemployment duration," Labour Economics, Elsevier, vol. 15(5), pages 1102-1125, October.
    10. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    11. James J. Heckman & Justin L. Tobias & Edward Vytlacil, 2000. "Simple Estimators for Treatment Parameters in a Latent Variable Framework with an Application to Estimating the Returns to Schooling," NBER Working Papers 7950, National Bureau of Economic Research, Inc.
    12. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    13. James Heckman, 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 441-462.
    14. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    15. James J. Heckman & Jeffrey A. Smith, 1998. "Evaluating the Welfare State," NBER Working Papers 6542, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Yonatan Eyal, 2020. "Self-Assessment Variables as a Source of Information in the Evaluation of Intervention Programs: A Theoretical and Methodological Framework," SAGE Open, , vol. 10(1), pages 21582440198, January.
    3. Słoczyński, Tymon, 2012. "New Evidence on Linear Regression and Treatment Effect Heterogeneity," MPRA Paper 39524, University Library of Munich, Germany.
    4. James J. Heckman, 2008. "Econometric Causality," International Statistical Review, International Statistical Institute, vol. 76(1), pages 1-27, April.
    5. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    6. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    7. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    8. James J. Heckman, 2008. "The Principles Underlying Evaluation Estimators with an Application to Matching," Annals of Economics and Statistics, GENES, issue 91-92, pages 9-73.
    9. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    10. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    11. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    12. Heckman, James J., 2010. "The Assumptions Underlying Evaluation Estimators," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 30(2), December.
    13. Anirban Basu & James J. Heckman & Salvador Navarro‐Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self‐selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157, November.
    14. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    15. Heckman, James, 2001. "Accounting for Heterogeneity, Diversity and General Equilibrium in Evaluating Social Programmes," Economic Journal, Royal Economic Society, vol. 111(475), pages 654-699, November.
    16. Cristian Aedo & Sergio Nuñez, 2004. "Efectos de las políticas de capacitación en América Latina y el Caribe: el caso del Programa Joven," Research Department Publications 3176, Inter-American Development Bank, Research Department.
    17. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    18. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    19. Elías, Víctor & Ruiz Núñez, Fernanda & Cossa, Ricardo & Bravo, David, 2004. "An Econometric Cost-Benefit Analysis of Argentina’s Youth Training Program," IDB Publications (Working Papers) 13053, Inter-American Development Bank.
    20. Thomas Andren & Daniela Andren, 2006. "Assessing the employment effects of vocational training using a one-factor model," Applied Economics, Taylor & Francis Journals, vol. 38(21), pages 2469-2486.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:34:y:2010:i:6:p:455-486. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.