IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0295.html

The Role of Randomized Field Trials in Social Science Research: A Perspective from Evaluations of Reforms of Social Welfare Programs

Author

Listed:
  • Robert A. Moffitt

Abstract

One of the areas of policy research where randomized field trials have been utilized most intensively is welfare reform. Starting in the late 1960s with experimental tests of a negative income tax and continuing through current experimental tests of recent welfare reforms, randomized evaluations have played a strong and increasing role in informing policy. This paper reviews the record of these experiments and assesses the implications of that record for the use of randomization. The review demonstrates that, while randomized field trials in the area of welfare reform have been professionally conducted and well-run, and have yielded much valuable and credible information, their usefulness has been limited by a number of weaknesses, some of which are inherent in the method and some of which result from constraints imposed by the political process. The conclusion is that randomized field trials have an important but limited role to play in future welfare reform evaluations, and that it is essential that they be supplemented by nonexperimental research.

Suggested Citation

  • Robert A. Moffitt, 2003. "The Role of Randomized Field Trials in Social Science Research: A Perspective from Evaluations of Reforms of Social Welfare Programs," NBER Technical Working Papers 0295, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberte:0295
    Note: TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0295.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Moffitt, Robert, 1992. "Incentive Effects of the U.S. Welfare System: A Review," Journal of Economic Literature, American Economic Association, vol. 30(1), pages 1-61, March.
    2. David Card & Winston Lin & Philip Robins, 1997. "Would Financial Incentives for Leaving Welfare Lead Some People to Stay on Welfare Longer? An Experimental Evaluation of 'Entry Effects' in the Self-Sufficiency Project," Working Papers 759, Princeton University, Department of Economics, Industrial Relations Section..
    3. Michael J. Camasso & Radha Jagannathan & Carol Harvey & Mark Killingsworth, 2003. "The use of client surveys to gauge the threat of contamination in welfare reform experiments," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 22(2), pages 207-223.
    4. Carol Harvey & Michael J. Camasso & Radha Jagannathan, 2000. "Evaluating Welfare Reform Waivers under Section 1115," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 165-188, Fall.
    5. David Card & Philip K. Robins & Winston Lin, 1998. "Would Financial Incentives for Leaving Welfare Lead Some People to Stay on Welfare Longer? An Experimental Evaluation of 'Entry Effects' in the SSP," NBER Working Papers 6449, National Bureau of Economic Research, Inc.
    6. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    7. repec:pri:indrel:dsp01rv042t06t is not listed on IDEAS
    8. Rebecca M. Blank, 2002. "Evaluating Welfare Reform in the United States," Journal of Economic Literature, American Economic Association, vol. 40(4), pages 1105-1166, December.
    9. Ashenfelter, Orley & Plant, Mark W, 1990. "Nonparametric Estimates of the Labor-Supply Effects of Negative Income Tax Programs," Journal of Labor Economics, University of Chicago Press, vol. 8(1), pages 396-415, January.
    10. V. Joseph Hotz & Guido W. Imbens & Julie H. Mortimer, 1999. "Predicting the Efficacy of Future Training Programs Using Past Experiences," NBER Technical Working Papers 0238, National Bureau of Economic Research, Inc.
    11. Robert A. Moffitt, 1996. "The effect of employment and training programs on entry and exit from the welfare caseload," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 15(1), pages 32-50.
    12. repec:fth:prinin:380 is not listed on IDEAS
    13. Gary Burtless, 1986. "The work response to a guaranteed income: a survey of experimental evidence," Conference Series ; [Proceedings], Federal Reserve Bank of Boston, vol. 30, pages 22-59.
    14. Alicia H. Munnell, 1987. "Lessons from the income maintenance experiments: an overview," New England Economic Review, Federal Reserve Bank of Boston, issue May, pages 32-45.
    15. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2000. "The Long-Term Gains from GAIN: A Re-Analysis of the Impacts of the California GAIN Program," NBER Working Papers 8007, National Bureau of Economic Research, Inc.
    16. David Card & Winston Lin & Philip Robins, 1997. "Would Financial Incentives for Leaving Welfare Lead Some People to Stay on Welfare Longer? An Experimental Evaluation of 'Entry Effects' in the Self-Sufficiency Project," Working Papers 759, Princeton University, Department of Economics, Industrial Relations Section..
    17. repec:fth:prinin:259 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Leventhal, Tama & Dupéré, Véronique, 2011. "Moving to Opportunity: Does long-term exposure to 'low-poverty' neighborhoods make a difference for adolescents?," Social Science & Medicine, Elsevier, vol. 73(5), pages 737-743, September.
    2. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    3. World Bank, 2007. "Poverty and Environment : Understanding Linkages at the Household Level," World Bank Publications - Reports 7744, The World Bank Group.
    4. Hugo Benítez-Silva & Richard Disney & Sergi Jiménez-Martín, 2010. "Disability, capacity for work and the business cycle: an international perspective [Has the boom in incapacity benefit claimant numbers passed its peak?]," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 25(63), pages 483-536.
    5. World Bank, 2007. "Poverty and Environment : Understanding Linkages at the Household Level," World Bank Publications - Books, The World Bank Group, number 6924, April.
    6. Kling, Jeffrey R., 2007. "Methodological Frontiers of Public Finance Field Experiments," National Tax Journal, National Tax Association;National Tax Journal, vol. 60(1), pages 109-127, March.
    7. Hugo Benitez-Silva & Moshe Buchinsky & John Rust, 2005. "Induced Entry Effects of a $1 for $2 Offset in SSDI Benefits," Department of Economics Working Papers 05-03, Stony Brook University, Department of Economics.
    8. Ted Joyce & Robert Kaestner & Sanders Korenman & Stanley Henshaw, 2004. "Family Cap Provisions and Changes in Births and Abortions," NBER Working Papers 10214, National Bureau of Economic Research, Inc.
    9. Nagy, Gyula & Micklewright, John, 2006. "Az álláskeresés ellenőrzése és a munkanélküliség időtartama. Egy társadalomtudományi kísérlet [Job-search monitoring and unemployment duration: evidence from a randomized control trial]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(7), pages 641-660.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rebecca M. Blank, 2002. "Evaluating Welfare Reform in the United States," Journal of Economic Literature, American Economic Association, vol. 40(4), pages 1105-1166, December.
    2. Bernard Fortin, 1997. "Dépendance à l’égard de l’aide sociale et réforme de la sécurité du revenu," L'Actualité Economique, Société Canadienne de Science Economique, vol. 73(4), pages 557-573.
    3. Ted Joyce & Robert Kaestner & Sanders Korenman & Stanley Henshaw, 2004. "Family Cap Provisions and Changes in Births and Abortions," NBER Working Papers 10214, National Bureau of Economic Research, Inc.
    4. John V. Pepper, 2003. "Using Experiments to Evaluate Performance Standards: What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?," Journal of Human Resources, University of Wisconsin Press, vol. 38(4).
    5. Lemieux, Thomas & Milligan, Kevin, 2008. "Incentive effects of social assistance: A regression discontinuity approach," Journal of Econometrics, Elsevier, vol. 142(2), pages 807-828, February.
    6. Card, David & Robins, Philip K., 2005. "How important are "entry effects" in financial incentive programs for welfare recipients? Experimental evidence from the Self-Sufficiency Project," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 113-139.
    7. Donal O'Neill, 2000. "Evaluating Labour Market Interventions," Economics Department Working Paper Series n990300, Department of Economics, National University of Ireland - Maynooth.
    8. Dehejia, Rajeev H., 2005. "Program evaluation as a decision problem," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 141-173.
    9. Neumark, David & Powers, Elizabeth T., 2005. "SSI, Labor Supply, and Migration," IZA Discussion Papers 1820, Institute of Labor Economics (IZA).
    10. Dhaval Dave & Hope Corman & Nancy Reichman, 2012. "Effects of Welfare Reform on Education Acquisition of Adult Women," Journal of Labor Research, Springer, vol. 33(2), pages 251-282, June.
    11. L. M. Mead, "undated". "Welfare policy: The administrative frontier," Institute for Research on Poverty Discussion Papers 1093-96, University of Wisconsin Institute for Research on Poverty.
    12. Andrew Bibler & Mouhcine Guettabi & Matthew N. Reimer, 2023. "Universal Cash Transfers and Labor Market Outcomes," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 198-224, January.
    13. Ayala, Luis & Rodriguez, Magdalena, 2007. "Barriers to employment and welfare dynamics: Evidence from Spain," Journal of Policy Modeling, Elsevier, vol. 29(2), pages 237-257.
    14. Berthold, Norbert & von Berchem, Sascha, 2002. "Sozialhilfe im wettbewerblichen Föderalismus: Erfahrungen der USA, Lehren für Deutschland," Discussion Paper Series 58, Julius Maximilian University of Würzburg, Chair of Economic Order and Social Policy.
    15. Peter R. Mueser & Kenneth R. Troske & David R. Stevens, 2007. "The Impact of Welfare Reform on Leaver Characteristics, Employment and Recidivism: An Analysis of Maryland and Missouri," Working Papers 0720, Department of Economics, University of Missouri.
    16. Dhaval M. Dave & Nancy E. Reichman & Hope Corman, 2008. "Effects of Welfare Reform on Educational Acquisition of Young Adult Women," NBER Working Papers 14466, National Bureau of Economic Research, Inc.
    17. Martin Halla & Mario Lackner & Johann Scharler, 2016. "Does the Welfare State Destroy the Family? Evidence from OECD Member Countries," Scandinavian Journal of Economics, Wiley Blackwell, vol. 118(2), pages 292-323, April.
    18. Xavier Ramos & Timo Verlaat & Federico Todeschini, 2023. "The Employment Effects of Generous and Unconditional Cash Support," Working Papers 638, ECINEQ, Society for the Study of Economic Inequality.
    19. Adam Looney, 2005. "The effects of welfare reform and related policies on single mothers' welfare use and employment," Finance and Economics Discussion Series 2005-45, Board of Governors of the Federal Reserve System (U.S.).

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • I3 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0295. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.