IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0107.html
   My bibliography  Save this paper

Randomization and Social Policy Evaluation

Author

Listed:
  • James J. Heckman

Abstract

This paper considers the recent case for randomized social experimentation and contrasts it with older cases for social experimentation. The recent case eschews behavioral models, assumes that certain mean differences in outcomes are the parameters of interest to evaluators and assumes that randomization does not disrupt the social program being analyzed. Conditions under which program disruption effects are of no consequence are presented. Even in the absence of randomization bias, ideal experimental data cannot estimate median (other quantile) differences between treated and untreated persons without invoking supplementary statistical assumptions. The recent case for randomized experimentation does not address the choice of the appropriate stage in a multistage program at which randomization should be conducted. Evidence on randomization bias is presented.

Suggested Citation

  • James J. Heckman, 1991. "Randomization and Social Policy Evaluation," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberte:0107
    Note: LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0107.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Richard Arnott & Joseph E. Stiglitz, 1988. "Randomization with Asymmetric Information," RAND Journal of Economics, The RAND Corporation, vol. 19(3), pages 344-362, Autumn.
    3. Manski, C.F., 1990. "The Selection Problem," Working papers 90-12, Wisconsin Madison - Social Systems.
    4. Heckman, James J, 1978. "Dummy Endogenous Variables in a Simultaneous Equation System," Econometrica, Econometric Society, vol. 46(4), pages 931-959, July.
    5. Heckman, James J & Honore, Bo E, 1990. "The Empirical Content of the Roy Model," Econometrica, Econometric Society, vol. 58(5), pages 1121-1149, September.
    6. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-660, November.
    7. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    8. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
    9. Conlisk, John, 1973. "Choice of Response Functional Form in Designing Subsidy Experiments," Econometrica, Econometric Society, vol. 41(4), pages 643-656, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. History of policy evaluation: a few questions
      by Beatrice Cherrier in History of Economics Playground on 2015-02-05 21:26:43

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Orley Ashenfelter & David Ashmore & Olivier Deschenes, 1998. "Do Unemployment Insurance Recipients Actively Seek Work? Randomized Trials in Four U.S. States," Working Papers 791, Princeton University, Department of Economics, Industrial Relations Section..
    2. V. Joseph Hotz & Susan Williams McElroy & Seth G. Sanders, 2005. "Teenage Childbearing and Its Life Cycle Consequences: Exploiting a Natural Experiment," Journal of Human Resources, University of Wisconsin Press, vol. 40(3).
    3. Cockx, Bart & Ridder, Geert, 2001. "Social Employment of Welfare Recipients in Belgium: An Evaluation," Economic Journal, Royal Economic Society, vol. 111(470), pages 322-352, April.
    4. James Heckman & Jeffrey Smith & Christopher Taber, 1994. "Accounting for Dropouts in Evaluations of Social Experiments," NBER Technical Working Papers 0166, National Bureau of Economic Research, Inc.
    5. Arnab Acharya & Giulia Greco & Edoardo Masset, 2010. "The economics approach to evaluation of health interventions in developing countries through randomised field trial," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 2(4), pages 401-420.
    6. Courty, Pascal & Marschke, Gerald, 2004. "A General Test of Gaming," CEPR Discussion Papers 4514, C.E.P.R. Discussion Papers.
    7. Michael A. Boozer & Stephen E. Cacciola, 2001. "Inside the 'Black Box' of Project STAR: Estimation of Peer Effects Using Experimental Data," Working Papers 832, Economic Growth Center, Yale University.
    8. Florent Bedecarrats & Isabelle Guérin & François Roubaud, 2017. "L'étalon-or des évaluations randomisées : du discours de la méthode à l'économie politique," Working Papers ird-01445209, HAL.
    9. Cockx, Bart & Bardoulat, Isabelle, 1999. "Vocational Training: Does it speed up the Transition Rate out of Unemployment ?," Discussion Papers (IRES - Institut de Recherches Economiques et Sociales) 1999032, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    10. Mark Rosenzweig & Ahmed Musfiq Mobarak, 2013. "Risk, Insurance and Wages in General Equilibrium," Working Papers 1035, Economic Growth Center, Yale University.
    11. C. F. Manski, "undated". "Learning about social programs from experiments with random assignment of treatments," Institute for Research on Poverty Discussion Papers 1061-95, University of Wisconsin Institute for Research on Poverty.
    12. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    13. Markus Gangl & Thomas A. DiPrete, 2004. "Kausalanalyse durch Matchingverfahren," Discussion Papers of DIW Berlin 401, DIW Berlin, German Institute for Economic Research.
    14. Kluve, Jochen, 2001. "On the Role of Counterfactuals in Inferring Causal Effects of Treatments," IZA Discussion Papers 354, Institute for the Study of Labor (IZA).
    15. Cunha, Flavio & Heckman, James, 2008. "A New Framework For The Analysis Of Inequality," Macroeconomic Dynamics, Cambridge University Press, vol. 12(S2), pages 315-354, September.
    16. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2018. "The gold standard for randomised evaluations: from discussion of method to political economy," Working Papers hal-01686672, HAL.
    17. repec:mpr:mprres:6250 is not listed on IDEAS
    18. Peter Dolton; & Donal O'Neill, 1997. "The Long-Run Effects of Unemployment Monitoring and Work-Search Programs: Some Experimental Evidence from the U.K," Economics, Finance and Accounting Department Working Paper Series n710897, Department of Economics, Finance and Accounting, National University of Ireland - Maynooth.
    19. John V. Pepper, 1999. "What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?," JCPR Working Papers 105, Northwestern University/University of Chicago Joint Center for Poverty Research.
    20. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomised evaluations: from discussion of method to political economics," Working Papers CEB 15-009, ULB -- Universite Libre de Bruxelles.
    21. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    22. Gennetian, Lisa A. & Hill, Heather D. & London, Andrew S. & Lopoo, Leonard M., 2010. "Maternal employment and the health of low-income young children," Journal of Health Economics, Elsevier, vol. 29(3), pages 353-363, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0107. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.