IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

Partner Selection into Policy Relevant Field Experiments

This study investigates the issue of self-selection of stakeholders into participation and collaboration in policy-relevant experiments. We document and test the implications of self-selection in the context of randomised policy experiment we conducted in primary schools in the UK. The main questions we ask are (1) is there evidence of selection on key observable characteristics likely to matter for the outcome of interest and (2) does selection matter for the estimates of treatment effects. The experimental work consists in testing the effects of an intervention aimed at encouraging children to make more healthy choices at lunch. We recruited schools through local authorities and randomised schools across two incentive treatments and a control group. We document the selection taking place both at the level of local authorities and at the school level. Overall we find mild evidence of selection on key observables such as obesity levels and socio-economic characteristics. We find evidence of selection along indicators of involvement in healthy lifestyle programmes at the school level, but the magnitude is small. Moreover, we do not find significant differences in the treatment effects of the experiment between variables which, albeit to a mild degree, are correlated with selection into the experiment. To our knowledge, this is the first study providing direct evidence on the magnitude of self-selection in field experiments.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.econ.ed.ac.uk/papers/id236_esedps.pdf
Download Restriction: no

Paper provided by Edinburgh School of Economics, University of Edinburgh in its series ESE Discussion Papers with number 236.

as
in new window

Length: 32
Date of creation: 17 Dec 2013
Date of revision:
Handle: RePEc:edn:esedps:236
Contact details of provider: Postal:
31 Buccleuch Place, EH8 9JT, Edinburgh

Phone: +44(0)1316508361
Fax: +44(0)1316504514
Web page: http://www.econ.ed.ac.uk/

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Slonim, Robert & Wang, Carmen & Garbarino, Ellen & Merrett, Danielle, 2013. "Opting-in: Participation bias in economic experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 43-70.
  2. Pradhan, Menno & Suryadarma, Daniel & Beatty, Amanda & Wong, Maisy & Alishjabana, Armida & Gaduh, Arya & Artha, Rima Prama, 2011. "Improving educational quality through enhancing community participation : results from a randomized field experiment in Indonesia," Policy Research Working Paper Series 5795, The World Bank.
  3. Belot, Michele & James, J, 2014. "A New Perspective on the Issue of Selection Bias into Randomized Controlled Field Experiments," Department of Economics Working Papers 39843, University of Bath, Department of Economics.
  4. Belot, Michèle & James, Jonathan, 2009. "Healthy school meals and educational outcomes," ISER Working Paper Series 2009-01, Institute for Social and Economic Research.
  5. Glenn Harrison & John List, 2004. "Field experiments," Artefactual Field Experiments 00058, The Field Experiments Website.
  6. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
  7. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
  8. Jens Ludwig & Jeffrey R. Kling & Sendhil Mullainathan, 2011. "Mechanism Experiments and Policy Evaluations," Journal of Economic Perspectives, American Economic Association, vol. 25(3), pages 17-38, Summer.
  9. Ernst Fehr & Lorenz Goette, 2007. "Do Workers Work More if Wages Are High? Evidence from a Randomized Field Experiment," American Economic Review, American Economic Association, vol. 97(1), pages 298-317, March.
  10. Suresh De Mel & David McKenzie & Christopher Woodruff, 2012. "The demand for, and consequences of, formalization among informal firms in Sri Lanka," NBER Working Papers 18019, National Bureau of Economic Research, Inc.
  11. Raj Chetty & Adam Looney & Kory Kroft, 2009. "Salience and Taxation: Theory and Evidence," American Economic Review, American Economic Association, vol. 99(4), pages 1145-77, September.
  12. Jessica Wisdom & Julie S. Downs & George Loewenstein, 2010. "Promoting Healthy Choices: Information versus Convenience," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 164-78, April.
  13. Elisabet Rutstrom & Glenn Harrison & Morten Lau, 2004. "Estimating risk attitudes in denmark: A field experiment," Artefactual Field Experiments 00059, The Field Experiments Website.
  14. Orazio Attanasio & Britta Augsburg & Ralph De Haas & Emla Fitzsimons & Heike Harmgart, 2015. "The Impacts of Microfinance: Evidence from Joint-Liability Lending in Mongolia," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 90-122, January.
  15. Armin Falk, 2007. "Gift Exchange in the Field," Econometrica, Econometric Society, vol. 75(5), pages 1501-1511, 09.
  16. Damon Jones, 2010. "Information, Preferences, and Public Benefit Participation: Experimental Evidence from the Advance EITC and 401(k) Savings," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 147-63, April.
  17. Imran Rasul & Iwan Barankay & Orana Bandiera, 2006. "Incentives for managers and inequality among workers: Evidence from a firm level experiment," Natural Field Experiments 00213, The Field Experiments Website.
  18. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-74, August.
  19. Duflo, Esther & Gale, William & Liebman, Jeff & Orszag, Peter & Saez, Emmanuel, 2005. "Saving Incentives for Low- and Middle-Income Families: Evidence from a Field Experiment with H&R Block," CEPR Discussion Papers 5332, C.E.P.R. Discussion Papers.
  20. Henrik Jacobsen Kleven & Martin B. Knudsen & Claus Thustrup Kreiner & Søren Pedersen & Emmanuel Saez, 2011. "Unwilling or Unable to Cheat? Evidence From a Tax Audit Experiment in Denmark," Econometrica, Econometric Society, vol. 79(3), pages 651-692, 05.
  21. Blair Cleave & Nikos Nikiforakis & Robert Slonim, 2013. "Is there selection bias in laboratory experiments? The case of social and risk preferences," Experimental Economics, Springer, vol. 16(3), pages 372-382, September.
  22. Xavier Gine & Jessica Goldberg & Dean Yang, 2012. "Credit Market Consequences of Improved Personal Identification: Field Experimental Evidence from Malawi," American Economic Review, American Economic Association, vol. 102(6), pages 2923-54, October.
  23. John A. List, 2011. "Why Economists Should Conduct Field Experiments and 14 Tips for Pulling One Off," Journal of Economic Perspectives, American Economic Association, vol. 25(3), pages 3-16, Summer.
  24. Vivi Alatas & Abhijit Banerjee & Rema Hanna & Benjamin A. Olken & Julia Tobias, 2012. "Targeting the Poor: Evidence from a Field Experiment in Indonesia," American Economic Review, American Economic Association, vol. 102(4), pages 1206-40, June.
  25. Eric P. Bettinger & Bridget Terry Long & Philip Oreopoulos & Lisa Sanbonmatsu, 2012. "The Role of Application Assistance and Information in College Decisions: Results from the H&R Block Fafsa Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 127(3), pages 1205-1242.
  26. David Card & Stefano DellaVigna & Ulrike Malmendier, 2011. "The Role of Theory in Field Experiments," NBER Working Papers 17047, National Bureau of Economic Research, Inc.
  27. Malani, Anup, 2008. "Patient enrollment in medical trials: Selection bias in a randomized experiment," Journal of Econometrics, Elsevier, vol. 144(2), pages 341-351, June.
  28. Francesco Avvisati & Marc Gurgand & Nina Guyon & Eric Maurin, 2014. "Getting Parents Involved: A Field Experiment in Deprived Schools," Review of Economic Studies, Oxford University Press, vol. 81(1), pages 57-83.
  29. Belot, Michèle & James, Jonathan & Nolen, Patrick J., 2015. "Incentives and Children's Dietary Choices: A Field Experiment in Primary Schools," IZA Discussion Papers 9424, Institute for the Study of Labor (IZA).
  30. Esther Duflo & Michael Greenstone & Nicholas Ryan, 2013. "Truth-telling by Third-party Auditors and the Response of Polluting Firms: Experimental Evidence from India," The Quarterly Journal of Economics, Oxford University Press, vol. 128(4), pages 1499-1545.
  31. Hongbin Cai & Yuyu Chen & Hanming Fang, 2007. "Observational Learning: Evidence from a Randomized Natural Field Experiment," NBER Working Papers 13516, National Bureau of Economic Research, Inc.
  32. Bruce Shearer, 2004. "Piece Rates, Fixed Wages and Incentives: Evidence from a Field Experiment," Review of Economic Studies, Oxford University Press, vol. 71(2), pages 513-534.
  33. Dean Karlan & John A. List, 2006. "Does Price Matter in Charitable Giving? Evidence from a Large-Scale Natural Field Experiment," Working Papers 1, The Field Experiments Website.
  34. Kate Ambler & Diego Aycinena & Dean Yang, 2014. "Channeling Remittances to Education: A Field Experiment Among Migrants from El Salvador," NBER Working Papers 20262, National Bureau of Economic Research, Inc.
  35. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097 Elsevier.
  36. Joseph Hotz, V. & Imbens, Guido W. & Mortimer, Julie H., 2005. "Predicting the efficacy of future training programs using past experiences at other locations," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 241-270.
  37. Daniel Nagin & James Rebitzer & Seth Sanders & Lowell Taylor, 2002. "Monitoring, Motivation and Management: The Determinants of Opportunistic Behavior in a Field Experiment," NBER Working Papers 8811, National Bureau of Economic Research, Inc.
  38. Roland G. Fryer, 2011. "Financial Incentives and Student Achievement: Evidence from Randomized Trials," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1755-1798.
  39. Emily Oster & Rebecca Thornton, 2011. "Menstruation, Sanitary Products, and School Attendance: Evidence from a Randomized Evaluation," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 91-100, January.
  40. repec:feb:artefa:0110 is not listed on IDEAS
  41. Pascaline Dupas & Jonathan Robinson, 2011. "Why Don't the Poor Save More? Evidence from Health Savings Experiments," NBER Working Papers 17255, National Bureau of Economic Research, Inc.
  42. Leonardo Bursztyn & Robert Jensen, 2015. "How Does Peer Pressure Affect Educational Investments?," The Quarterly Journal of Economics, Oxford University Press, vol. 130(3), pages 1329-1367.
  43. Frijters, Paul & Kong, Tao Sherry & Liu, Elaine M., 2015. "Who is coming to the artefactual field experiment? Participation bias among Chinese rural migrants," Journal of Economic Behavior & Organization, Elsevier, vol. 114(C), pages 62-74.
  44. Pieter A. Gautier & Bas van der Klaauw, 2012. "Selection in a field experiment with voluntary participation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(1), pages 63-84, 01.
  45. Esther Duflo & Rachel Glennerster & Michael Kremer, 2006. "Using Randomization in Development Economics Research: A Toolkit," NBER Technical Working Papers 0333, National Bureau of Economic Research, Inc.
  46. Moussa P. Blimpo, 2014. "Team Incentives for Education in Developing Countries: A Randomized Field Experiment in Benin," American Economic Journal: Applied Economics, American Economic Association, vol. 6(4), pages 90-109, October.
  47. Erica Field & Rohini Pande & John Papp & Natalia Rigol, 2013. "Does the Classic Microfinance Model Discourage Entrepreneurship among the Poor? Experimental Evidence from India," American Economic Review, American Economic Association, vol. 103(6), pages 2196-2226, October.
  48. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, Oxford University Press, vol. 130(3), pages 1117-1165.
  49. Heather Royer & Mark Stehr & Justin Sydnor, 2015. "Incentives, Commitments, and Habit Formation in Exercise: Evidence from a Field Experiment with Workers at a Fortune-500 Company," American Economic Journal: Applied Economics, American Economic Association, vol. 7(3), pages 51-84, July.
  50. Lant Pritchett, 2002. "It pays to be ignorant: A simple political economy of rigorous program evaluation," Journal of Economic Policy Reform, Taylor & Francis Journals, vol. 5(4), pages 251-269.
  51. Justine S. Hastings & Jeffrey M. Weinstein, 2008. "Information, School Choice, and Academic Achievement: Evidence from Two Experiments," The Quarterly Journal of Economics, Oxford University Press, vol. 123(4), pages 1373-1414.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:edn:esedps:236. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Gina Reddie)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.