IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0262.html
   My bibliography  Save this paper

Treatment Effects for Discrete Outcomes when Responses to Treatment Vary Among Observationally Identical Persons: An Application to Norwegian ..

Author

Listed:
  • Arild Aakvik
  • James J. Heckman
  • Edward J. Vytlacil

Abstract

This paper formulates an econometric framework for studying the impact of interventions on discrete outcomes when responses to treatment vary among observationally identical persons. Using a latent variable model that can be linked to well-posed economic models, we show how to define and interpret the average treatment effects, the average effect of treatment on the treated, the marginal treatment effect and the distribution of treatment effects for discrete outcomes. To estimate these parameters and the distribution of treatment effects, we formulate and estimate a discrete choice model with unobservables generated by a factor structure model. We apply our methods to evaluate the effect of Norwegian Vocational Rehabilitation training programs on employment outcomes for women. We find that applicants to these programs who participate in active training have a 4.6% higher employment rate than nonparticipants. When we control for the observable characteristics of applicants, we find that the average treatment effects falls to 4.1%. When we control for the unobservables characteristics of applicants, the average treatment effect falls to -1.4% and effect of treatment on the treated is -11%. We also find evidence of substantial heterogeneity in response to training.

Suggested Citation

  • Arild Aakvik & James J. Heckman & Edward J. Vytlacil, 2000. "Treatment Effects for Discrete Outcomes when Responses to Treatment Vary Among Observationally Identical Persons: An Application to Norwegian ..," NBER Technical Working Papers 0262, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberte:0262
    Note: TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0262.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Heckman, James & Singer, Burton, 1984. "A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data," Econometrica, Econometric Society, vol. 52(2), pages 271-320, March.
    2. Heckman, James J & Lochner, Lance & Taber, Christopher, 1998. "General-Equilibrium Treatment Effects: A Study of Tuition Policy," American Economic Review, American Economic Association, vol. 88(2), pages 381-386, May.
    3. Joshua D. Angrist & Kathryn Graddy & Guido W. Imbens, 1995. "Non-Parametric Demand Analysis with an Application to the Demand for Fish," NBER Technical Working Papers 0178, National Bureau of Economic Research, Inc.
    4. Burt S. Barnow, 1987. "The Impact of CETA Programs on Earnings: A Review of the Literature," Journal of Human Resources, University of Wisconsin Press, vol. 22(2), pages 157-193.
    5. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    6. Heckman, James J. & Vytlacil, Edward J., 2000. "The relationship between treatment parameters within a latent variable framework," Economics Letters, Elsevier, vol. 66(1), pages 33-39, January.
    7. Heckman, James J & Honore, Bo E, 1990. "The Empirical Content of the Roy Model," Econometrica, Econometric Society, vol. 58(5), pages 1121-1149, September.
    8. Angrist, Joshua D, 2001. "Estimations of Limited Dependent Variable Models with Dummy Endogenous Regressors: Simple Strategies for Empirical Practice," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(1), pages 2-16, January.
    9. Ham, John C & LaLonde, Robert J, 1996. "The Effect of Sample Selection and Initial Conditions in Duration Models: Evidence from Experimental Data on Training," Econometrica, Econometric Society, vol. 64(1), pages 175-205, January.
    10. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    11. Xiaohong Chen & James J. Heckman & Edward Vytlacil, 2000. "Identification and SQRT N Efficient Estimation of Semiparametric Panel Data Models with Binary Dependent Variables and a Latent Factor," Econometric Society World Congress 2000 Contributed Papers 1567, Econometric Society.
    12. James J. Heckman & Edward J. Vytlacil, 2000. "Local Instrumental Variables," NBER Technical Working Papers 0252, National Bureau of Economic Research, Inc.
    13. Stephen H. Bell & Larry l. Orr & John D. Blomquist & Glen G. Cain, 1995. "Program Applicants as a Comparison Group in Evaluating Training Programs: Theory and a Test," Books from Upjohn Press, W.E. Upjohn Institute for Employment Research, number pacg, january-j.
    14. Butler, J S & Moffitt, Robert, 1982. "A Computationally Efficient Quadrature Procedure for the One-Factor Multinomial Probit Model," Econometrica, Econometric Society, vol. 50(3), pages 761-764, May.
    15. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    16. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    17. James Heckman & Lance Lochner & Christopher Taber, 1998. "Explaining Rising Wage Inequality: Explanations With A Dynamic General Equilibrium Model of Labor Earnings With Heterogeneous Agents," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(1), pages 1-58, January.
    18. Card, David & Sullivan, Daniel G, 1988. "Measuring the Effect of Subsidized Training Programs on Movements in and out of Employment," Econometrica, Econometric Society, vol. 56(3), pages 497-530, May.
    19. Gritz, R. Mark, 1993. "The impact of training on the frequency and duration of employment," Journal of Econometrics, Elsevier, vol. 57(1-3), pages 21-51.
    20. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    21. Ridder, G, 1986. "An Event History Approach to the Evaluation of Training, Recruitment and Employment Programmes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 1(2), pages 109-126, April.
    22. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
    23. Robert J. LaLonde, 1995. "The Promise of Public Sector-Sponsored Training Programs," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 149-168, Spring.
    24. A. D. Roy, 1951. "Some Thoughts On The Distribution Of Earnings," Oxford Economic Papers, Oxford University Press, vol. 3(2), pages 135-146.
    25. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    26. Poirier, Dale J., 1980. "Partial observability in bivariate probit models," Journal of Econometrics, Elsevier, vol. 12(2), pages 209-217, February.
    27. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aakvik, Arild & Salvanes, Kjell G & Vaage, Kjell, 2003. "Measuring Heterogeneity in the Returns to Education in Norway Using Educational Reforms," CEPR Discussion Papers 4088, C.E.P.R. Discussion Papers.
    2. Ayuya, Oscar I. & Gido, Eric O. & Bett, Hillary K. & Lagat, Job K. & Kahi, Alexander K. & Bauer, Siegfried, 2015. "Effect of Certified Organic Production Systems on Poverty among Smallholder Farmers: Empirical Evidence from Kenya," World Development, Elsevier, vol. 67(C), pages 27-37.
    3. Coelli, Michael B. & Green, David A. & Warburton, William P., 2007. "Breaking the cycle? The effect of education on welfare receipt among children of welfare recipients," Journal of Public Economics, Elsevier, vol. 91(7-8), pages 1369-1398, August.
    4. Anjana Susarla & Anitesh Barua, 2011. "Contracting Efficiency and New Firm Survival in Markets Enabled by Information Technology," Information Systems Research, INFORMS, vol. 22(2), pages 306-324, June.
    5. Pedro Carneiro, 2003. "Evaluating Education Policies when the Rate of Return Varies Across Individuals," Latin American Journal of Economics-formerly Cuadernos de Economía, Instituto de Economía. Pontificia Universidad Católica de Chile., vol. 40(121), pages 516-529.
    6. Min, Shi & Liu, Min & Huang, Jikun, 2020. "Does the application of ICTs facilitate rural economic transformation in China? Empirical evidence from the use of smartphones among farmers," Journal of Asian Economics, Elsevier, vol. 70(C).
    7. Julius Manda & Cornelis Gardebroek & Makaiko Khonje & Arega Alene & Munyaradzi Mutenje & Menale Kassie, 2016. "Determinants of child nutritional status in the eastern province of Zambia: the role of improved maize varieties," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 8(1), pages 239-253, February.
    8. Falk, Armin & Lalive, Rafael & Zweimüller, Josef, 2005. "The success of job applications: a new approach to program evaluation," Labour Economics, Elsevier, vol. 12(6), pages 739-748, December.
    9. Thomas Andren & Daniela Andren, 2006. "Assessing the employment effects of vocational training using a one-factor model," Applied Economics, Taylor & Francis Journals, vol. 38(21), pages 2469-2486.
    10. Fan, Yanqin & Park, Sang Soo, 2009. "Partial identification of the distribution of treatment effects and its confidence sets," MPRA Paper 37148, University Library of Munich, Germany.
    11. Melayne Morgan McInnes & Orgul Demet Ozturk & Suzanne McDermott & Joshua R. Mann, 2010. "Does supported employment work?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 29(3), pages 506-525.
    12. Michael Lokshin & Zurab Sajaia, 2011. "Impact of interventions on discrete outcomes: Maximum likelihood estimation of the binary choice models with binary endogenous regressors," Stata Journal, StataCorp LP, vol. 11(3), pages 368-385, September.
    13. Aakvik, Arild & Holmas, Tor Helge & Kjerstad, Egil, 2003. "A low-key social insurance reform--effects of multidisciplinary outpatient treatment for back pain patients in Norway," Journal of Health Economics, Elsevier, vol. 22(5), pages 747-762, September.
    14. Andrén , Daniela & Andrén, Thomas, 2002. "Assessing the Employment Effects of Labor Market Training Programs in Sweden," Working Papers in Economics 70, University of Gothenburg, Department of Economics.
    15. Geoff Perry & Tim Maloney, 2008. "Economic Evaluation of the Training Opportunities Programme in New Zealand," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 11(2), pages 163-185.
    16. Viktor Steiner & Tobias Hagen, 2002. "Was kann die Aktive Arbeitsmarktpolitik in Deutschland aus der Evaluationsforschung in anderen europäischen Ländern lernen?," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 3(2), pages 189-206, May.
    17. Zhang, Tao, 2003. "Identifying treatment effects of active labour market programmes for Norwegian adults," Memorandum 26/2003, Oslo University, Department of Economics.
    18. Philipp Eisenhauer & James J. Heckman & Edward Vytlacil, 2015. "The Generalized Roy Model and the Cost-Benefit Analysis of Social Programs," Journal of Political Economy, University of Chicago Press, vol. 123(2), pages 413-443.
    19. Ragui Assaad & Deborah Levison & Nadia Zibani, 2001. "The Effect of Child Work on School Enrollment in Egypt," Working Papers 0111, Economic Research Forum, revised 04 Dec 2001.
    20. Leurcharusmee, Supanika & Sirisrisakulchai, Jirakom & Suriya, Komsan & Keesookpun, Chutipong & Srinuan, Pratompong, 2017. "Fixed-to-Mobile Substitution: Effects of Mobile Broadband Subscription on Fixed Broadband Termination," 14th ITS Asia-Pacific Regional Conference, Kyoto 2017: Mapping ICT into Transformation for the Next Information Society 168513, International Telecommunications Society (ITS).
    21. Tesfaye, Wondimagegn & Tirivayi, Nyasha, 2016. "The effect of improved storage innovations on food security and welfare in Ethiopia," MERIT Working Papers 2016-063, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    22. Tesfaye, Wondimagegn & Tirivayi, Nyasha, 2018. "The impacts of postharvest storage innovations on food security and welfare in Ethiopia," Food Policy, Elsevier, vol. 75(C), pages 52-67.
    23. Lokshin, Michael & Glinskaya, Elena, 2008. "The effect of male migration for work on employment patterns of females in nepal," Policy Research Working Paper Series 4757, The World Bank.
    24. Mark Dusheiko & Hugh Gravelle & Rowena Jacobs, 2004. "The effect of practice budgets on patient waiting times: allowing for selection bias," Health Economics, John Wiley & Sons, Ltd., vol. 13(10), pages 941-958, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aakvik, Arild & Heckman, James J. & Vytlacil, Edward J., 2005. "Estimating treatment effects for discrete outcomes when responses to treatment vary: an application to Norwegian vocational rehabilitation programs," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 15-51.
    2. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    3. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    4. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    5. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    6. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    7. Michael Lechner, 2000. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," Journal of Human Resources, University of Wisconsin Press, vol. 35(2), pages 347-375.
    8. Hujer, Reinhard & Wellner, Marc, 2000. "The Effects of Public Sector Sponsored Training on Individual Employment Performance in East Germany," IZA Discussion Papers 141, Institute of Labor Economics (IZA).
    9. Hidehiko Ichimura & Christopher R. Taber, 2000. "Direct Estimation of Policy Impacts," NBER Technical Working Papers 0254, National Bureau of Economic Research, Inc.
    10. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    11. Cockx, Bart & Bardoulat, Isabelle, 1999. "Vocational Training: Does it speed up the Transition Rate out of Unemployment ?," LIDAM Discussion Papers IRES 1999032, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    12. R. Hujer & K.-O. Maurer & M. Wellner, 1997. "The Impact of Training on Unemployment Duration in West Germany -Combining a Discrete Hazard Rate Model with Matching Techniques-," Econometrics 9705001, University Library of Munich, Germany.
    13. Heckman, James, 2001. "Accounting for Heterogeneity, Diversity and General Equilibrium in Evaluating Social Programmes," Economic Journal, Royal Economic Society, vol. 111(475), pages 654-699, November.
    14. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    15. Deborah A. Cobb‐Clark & Thomas Crossley, 2003. "Econometrics for Evaluations: An Introduction to Recent Developments," The Economic Record, The Economic Society of Australia, vol. 79(247), pages 491-511, December.
    16. Thomas Brodaty & Bruno Crépon & Denis Fougère, 2007. "Les méthodes micro-économétriques d'évaluation et leurs applications aux politiques actives de l'emploi," Economie & Prévision, La Documentation Française, vol. 0(1), pages 93-118.
    17. Mingliang Li & Dale J. Poirier & Justin L. Tobias, 2004. "Do dropouts suffer from dropping out? Estimation and prediction of outcome gains in generalized selection models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 19(2), pages 203-225.
    18. Jaap Abbring & James Heckman, 2008. "Dynamic policy analysis," CeMMAP working papers CWP05/08, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    19. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    20. Aakvik, Arild & Holmas, Tor Helge & Kjerstad, Egil, 2003. "A low-key social insurance reform--effects of multidisciplinary outpatient treatment for back pain patients in Norway," Journal of Health Economics, Elsevier, vol. 22(5), pages 747-762, September.

    More about this item

    JEL classification:

    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General
    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0262. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.