IDEAS home Printed from
   My bibliography  Save this paper

Analyzing Social Experiments as Implemented: A Reexamination of the Evidence from the HighScope Perry Preschool Program


  • Heckman, James J.

    () (University of Chicago)

  • Moon, Seong Hyeok

    () (University of Chicago)

  • Pinto, Rodrigo

    () (University of California, Los Angeles)

  • Savelyev, Peter A.

    () (College of William and Mary)

  • Yavitz, Adam

    () (University of Chicago)


Social experiments are powerful sources of information about the effectiveness of interventions. In practice, initial randomization plans are almost always compromised. Multiple hypotheses are frequently tested. "Significant" effects are often reported with p-values that do not account for preliminary screening from a large candidate pool of possible effects. This paper develops tools for analyzing data from experiments as they are actually implemented. We apply these tools to analyze the influential HighScope Perry Preschool Program. The Perry program was a social experiment that provided preschool education and home visits to disadvantaged children during their preschool years. It was evaluated by the method of random assignment. Both treatments and controls have been followed from age 3 through age 40. Previous analyses of the Perry data assume that the planned randomization protocol was implemented. In fact, as in many social experiments, the intended randomization protocol was compromised. Accounting for compromised randomization, multiple-hypothesis testing, and small sample sizes, we find statistically significant and economically important program effects for both males and females. We also examine the representativeness of the Perry study.

Suggested Citation

  • Heckman, James J. & Moon, Seong Hyeok & Pinto, Rodrigo & Savelyev, Peter A. & Yavitz, Adam, 2010. "Analyzing Social Experiments as Implemented: A Reexamination of the Evidence from the HighScope Perry Preschool Program," IZA Discussion Papers 5095, Institute for the Study of Labor (IZA).
  • Handle: RePEc:iza:izadps:dp5095

    Download full text from publisher

    File URL:
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    1. John Micklewright & Gyula Nagy, 1997. "The Implications of Exhausting Unemployment Insurance Entitlement in Hungary," Papers iopeps97/8, Innocenti Occasional Papers, Economic Policy Series.
    2. B Bell & Stephen Nickell, 1996. "Would Cutting Payroll Taxes on the Unskilled Have a Significant Effect on Unemployment?," CEP Discussion Papers dp0276, Centre for Economic Performance, LSE.
    3. Niall O'Higgins, 2005. "Trends in the Youth Labour Market in Developing and Transition Countries," Labor and Demography 0507002, EconWPA.
    4. L.Guarcello & M. Manacorda & F. Rosati & J. Fares & S.Lyon & C. Valdivia, 2005. "School-to-Work Transitions in Sub-Saharan Africa: An overview," UCW Working Paper 15, Understanding Children's Work (UCW Programme).
    5. John Van Reenen, 2004. "Active Labor Market Policies and the British New Deal for the Young Unemployed in Context," NBER Chapters,in: Seeking a Premier Economy: The Economic Effects of British Economic Reforms, 1980-2000, pages 461-496 National Bureau of Economic Research, Inc.
    6. Barbara Sianesi & John Van Reenen, 2003. "The Returns to Education: Macroeconomics," Journal of Economic Surveys, Wiley Blackwell, vol. 17(2), pages 157-200, April.
    7. World Bank, 2005. "Expanding Opportunities and Building Competencies for Young People : A New Agenda for Secondary Education," World Bank Publications, The World Bank, number 7429, March.
    8. Kertesi, Gabor & Köll?, János, 2003. "Fighting “Low Equilibria” by Doubling the Minimum Wage? Hungary’s Experiment," IZA Discussion Papers 970, Institute for the Study of Labor (IZA).
    9. Paul Ryan, 2001. "The School-to-Work Transition: A Cross-National Perspective: Corrigendum," Journal of Economic Literature, American Economic Association, vol. 39(3), pages 767-767, September.
    10. O'Higgins, Niall, 2015. "Youth Unemployment," IZA Policy Papers 103, Institute for the Study of Labor (IZA).
    11. Robert Shimer, 2001. "The Impact of Young Workers on the Aggregate Labor Market," The Quarterly Journal of Economics, Oxford University Press, vol. 116(3), pages 969-1007.
    12. Andrea Bassanini & Romain Duval, 2006. "Employment Patterns in OECD Countries: Reassessing the Role of Policies and Institutions," OECD Social, Employment and Migration Working Papers 35, OECD Publishing.
    13. Martina Lubyova & Jan C. van Ours, 1999. "Unemployment durations of job losers in a labour market in transition," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 7(3), pages 665-686, November.
    14. Martin Godfrey, 2003. "Youth employment policy in developing and transition countries - preventionas well as cure," Social Protection and Labor Policy and Technical Notes 27875, The World Bank.
    15. Friedrich G. Schneider, 2006. "Shadow Economies and Corruption all over the World: What do we really know?," Economics working papers 2006-17, Department of Economics, Johannes Kepler University Linz, Austria.
    16. Tito Boeri & Katherine Terrell, 2002. "Institutional Determinants of Labor Reallocation in Transition," Journal of Economic Perspectives, American Economic Association, vol. 16(1), pages 51-76, Winter.
    17. Niall O'Higgins & Andrey Ivanov, 2006. "Education and Employment Opportunities for the Roma," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 48(1), pages 6-19, March.
    18. Rutkowski, Jan, 2006. "Labor market developments during economic transition," Policy Research Working Paper Series 3894, The World Bank.
    19. O'Higgins, Niall, 2001. "Youth unemployment and employment policy: a global perspective," MPRA Paper 23698, University Library of Munich, Germany.
    20. Sandrine Cazes & Stefano Scarpetta, 1998. "Labour market transitions and unemployment duration: Evidence from Bulgarian and Polish micro-data," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 6(1), pages 113-144, May.
    21. Jan J. Rutkowski & Stefano Scarpetta, 2005. "Enhancing Job Opportunities : Eastern Europe and the Former Soviet Union," World Bank Publications, The World Bank, number 7408, March.
    22. Betcherman, Gordon & Olivas, Karina & Dar, Amit, 2004. "Impacts of active labor market programs : new evidence from evaluations with particular attention to developing and transition countries," Social Protection and Labor Policy and Technical Notes 29142, The World Bank.
    23. Sanders Korenman & David Neumark, 2000. "Cohort Crowding and Youth Labor Markets (A Cross-National Analysis)," NBER Chapters,in: Youth Employment and Joblessness in Advanced Countries, pages 57-106 National Bureau of Economic Research, Inc.
    24. Fretwell, David H. & Benus, Jacob & O'Leary, Christopher J., 1999. "Evaluating the impact of active labor programs : results of cross country studies in Europe and Central Asia," Social Protection and Labor Policy and Technical Notes 20131, The World Bank.
    25. Mukesh Chawla & Gordon Betcherman & Arup Banerji, 2007. "From Red to Gray : The "Third Transition" of Aging Populations in Eastern Europe and the Former Soviet Union," World Bank Publications, The World Bank, number 6741, March.
    26. Robert G. Fay, 1996. "Enhancing the Effectiveness of Active Labour Market Policies: Evidence from Programme Evaluations in OECD Countries," OECD Labour Market and Social Policy Occasional Papers 18, OECD Publishing.
    27. Paul Ryan, 2001. "The School-to-Work Transition: A Cross-National Perspective," Journal of Economic Literature, American Economic Association, vol. 39(1), pages 34-92, March.
    28. Canning, M. & Moock, P. & Heleniak, T., 1999. "Reforming Education in the Regions of Russia," Papers 457, World Bank - Technical Papers.
    Full references (including those not matched with items on IDEAS)

    More about this item


    social experiment; compromised randomization; early childhood intervention; multiple-hypothesis testing;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • J15 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Minorities, Races, Indigenous Peoples, and Immigrants; Non-labor Discrimination

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp5095. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.