IDEAS home Printed from https://ideas.repec.org/p/ifs/cemmap/22-10.html
   My bibliography  Save this paper

Analyzing social experiments as implemented: evidence from the HighScope Perry Preschool Program

Author

Listed:
  • James Heckman

    () (Institute for Fiscal Studies and University of Chicago)

  • Seong Hyeok Moon

    (Institute for Fiscal Studies)

  • Rodrigo Pinto

    (Institute for Fiscal Studies)

  • Peter Savelyev

    (Institute for Fiscal Studies)

  • Adam Yavitz

    (Institute for Fiscal Studies)

Abstract

Social experiments are powerful sources of information about the effectiveness of interventions. In practice, initial randomization plans are almost always compromised. Multiple hypotheses are frequently tested. "Significant" effects are often reported with p-values that do not account for preliminary screening from a large candidate pool of possible effects. This paper develops tools for analyzing data from experiments as they are actually implemented. We apply these tools to analyze the influential HighScope Perry Preschool Program. The Perry program was a social experiment that provided preschool education and home visits to disadvantaged children during their preschool years. It was evaluated by the method of random assignment. Both treatments and controls have been followed from age 3 through age 40. Previous analyses of the Perry data assume that the planned randomization protocol was implemented. In fact, as in many social experiments, the intended randomization protocol was compromised. Accounting for compromised randomization, multiple-hypothesis testing, and small sample sizes, we find statistically significant and economically important program effects for both males and females. We also examine the representativeness of the Perry study. Download appendix

Suggested Citation

  • James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: evidence from the HighScope Perry Preschool Program," CeMMAP working papers CWP22/10, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  • Handle: RePEc:ifs:cemmap:22/10
    as

    Download full text from publisher

    File URL: http://cemmap.ifs.org.uk/wps/cwp2210.pdf
    Download Restriction: no

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Heckman, James J. & Kautz, Tim, 2012. "Hard evidence on soft skills," Labour Economics, Elsevier, vol. 19(4), pages 451-464.
    2. repec:ucp:jlabec:doi:10.1086/690652 is not listed on IDEAS
    3. Peter A. Savelyev, 2014. "Psychological Skills, Education, and Longevity of High-Ability Individuals," Vanderbilt University Department of Economics Working Papers 14-00007, Vanderbilt University Department of Economics.
    4. Orla Doyle, 2017. "The First 2,000 Days and Child Skills: Evidence from a Randomized Experiment of Home Visiting," Working Papers 201715, School of Economics, University College Dublin.
    5. repec:hrv:faseco:30749606 is not listed on IDEAS
    6. Marigen Narea, 2014. "Does early centre-based care have an impact on child cognitive and socio-emotional development? Evidence from Chile," CASE Papers case183, Centre for Analysis of Social Exclusion, LSE.
    7. Costas Meghir & Orazio Attanasio & Sarah Cattan & Emla Fitzsimons & Marta Rubio-Codina, 2015. "Estimating the Production Function for Human Capital: Results from a Randomized Control Trial in Colombia," Working Papers 1046, Economic Growth Center, Yale University.
    8. Marianne P. Bitler & Hilary W. Hoynes & Thurston Domina, 2014. "Experimental Evidence on Distributional Effects of Head Start," NBER Working Papers 20434, National Bureau of Economic Research, Inc.
    9. Pedro Carneiro & Katrine V. Løken & Kjell G. Salvanes, 2015. "A Flying Start? Maternity Leave Benefits and Long-Run Outcomes of Children," Journal of Political Economy, University of Chicago Press, vol. 123(2), pages 365-412.
    10. Eiji Yamamura & Yoshiro Tsutsui, 2017. "Trade policy preference, childhood sporting experience, and informal school curriculum: Examination from the viewpoint of behavioral economics," Discussion Papers in Economics and Business 17-25, Osaka University, Graduate School of Economics and Osaka School of International Public Policy (OSIPP).
    11. David Johnston & Carol Propper & Stephen Pudney & Michael Shields, 2014. "Child Mental Health And Educational Attainment: Multiple Observers And The Measurement Error Problem," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(6), pages 880-900, September.
    12. Peter Savelyev & Kegon Tan, 2015. "Socioemotional Skills, Education, and Health-Related Outcomes of High-Ability Individuals," Vanderbilt University Department of Economics Working Papers 15-00007, Vanderbilt University Department of Economics.
    13. Orla Doyle & Nick Fitzpatrick & Judy Lovett & Caroline Rawdon, 2015. "Early intervention and child health: Evidence from a Dublin-based randomized controlled trial," Working Papers 201505, Geary Institute, University College Dublin.
    14. Malte Sandner, 2015. "Effects of Early Childhood Intervention on Fertility and Maternal Employment: Evidence from a Randomized Controlled Trial," SOEPpapers on Multidisciplinary Panel Data Research 799, DIW Berlin, The German Socio-Economic Panel (SOEP).
    15. Fletcher, Jason M. & Wolfe, Barbara, 2016. "The importance of family income in the formation and evolution of non-cognitive skills in childhood," Economics of Education Review, Elsevier, vol. 54(C), pages 143-154.
    16. Holford, Angus & Pudney, Stephen, 2014. "Survey design and the determinants of subjective wellbeing: an experimental analysis," Understanding Society Working Paper Series 2014-06, Understanding Society at the Institute for Social and Economic Research.
    17. repec:cep:sticas:/183 is not listed on IDEAS
    18. John List & Azeem Shaikh & Yang Xu, 2016. "Multiple Hypothesis Testing in Experimental Economics," Artefactual Field Experiments 00402, The Field Experiments Website.
    19. Pietro Biroli, 2016. "Health and skill formation in early childhood," UBSCENTER - Working Papers 017, UBS International Center of Economics in Society - Department of Economics - University of Zurich.
    20. Michael J. Kottelenberg & Steven F. Lehrer, 2017. "Targeted or Universal Coverage? Assessing Heterogeneity in the Effects of Universal Child Care," Journal of Labor Economics, University of Chicago Press, vol. 35(3), pages 609-653.
    21. Jorge Luis García & James J. Heckman & Anna L. Ziff, 2017. "Gender Differences in the Benefits of an Influential Early Childhood Program," NBER Working Papers 23412, National Bureau of Economic Research, Inc.
    22. Doyle, Orla & Fitzpatrick, Nick & Lovett, Judy & Rawdon, Caroline, 2015. "Early intervention and child physical health: Evidence from a Dublin-based randomized controlled trial," Economics & Human Biology, Elsevier, vol. 19(C), pages 224-245.
    23. Michael J. Kottelenberg & Steven F. Lehrer, 2017. "Does Quebec's Subsidized Child Care Policy Give Boys and Girls an Equal Start?," NBER Working Papers 23259, National Bureau of Economic Research, Inc.
    24. Brian Jacob & Jesse Rothstein, 2016. "The Measurement of Student Ability in Modern Assessment Systems," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 85-108, Summer.
    25. Ashok Kaul & Michael Wolf, 2014. "The (possible) effect of plain packaging on smoking prevalence in Australia: a trend analysis," ECON - Working Papers 165, Department of Economics - University of Zurich.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ifs:cemmap:22/10. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Emma Hyman). General contact details of provider: http://edirc.repec.org/data/cmifsuk.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.