IDEAS home Printed from https://ideas.repec.org/p/cca/wpaper/292.html
   My bibliography  Save this paper

Assignment procedure biases in randomized policy experiments

Author

Listed:
  • Gani Aldashev
  • Georg Kirchsteiger
  • Alexander Sebald

Abstract

Randomized controlled trials (RCT) have become a dominant empirical tool in applied economics. The internal validity of RCTs crucially depends on the (implicit) assumption that the procedure assigning subjects to treatment and control groups has no effect on behavior. We show theoretically that this assumption is violated when people are motivated by belief-dependent preferences and care about the intentions of others. The choice of assignment procedure influences subjects’ behavior and, consequently, the RCTs’ findings. Strikingly, even a credible and explicit randomization procedure does not guarantee an unbiased prediction of the impact of a general introduction of the policy.

Suggested Citation

  • Gani Aldashev & Georg Kirchsteiger & Alexander Sebald, 2012. "Assignment procedure biases in randomized policy experiments," Carlo Alberto Notebooks 292, Collegio Carlo Alberto.
  • Handle: RePEc:cca:wpaper:292
    as

    Download full text from publisher

    File URL: http://www.carloalberto.org/assets/working-papers/no.292.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    2. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    3. Esther Duflo & William Gale & Jeffrey Liebman & Peter Orszag & Emmanuel Saez, 2006. "Saving Incentives for Low- and Middle-Income Families: Evidence from a Field Experiment with H&R Block," The Quarterly Journal of Economics, Oxford University Press, vol. 121(4), pages 1311-1346.
    4. Michalopoulos, Charles & Robins, Philip K. & Card, David, 2005. "When financial work incentives pay for themselves: evidence from a randomized social experiment for welfare recipients," Journal of Public Economics, Elsevier, vol. 89(1), pages 5-29, January.
    5. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    6. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    7. Dufwenberg, Martin & Kirchsteiger, Georg, 2004. "A theory of sequential reciprocity," Games and Economic Behavior, Elsevier, vol. 47(2), pages 268-298, May.
    8. Claudio Ferraz & Frederico Finan, 2008. "Exposing Corrupt Politicians: The Effects of Brazil's Publicly Released Audits on Electoral Outcomes," The Quarterly Journal of Economics, Oxford University Press, vol. 123(2), pages 703-745.
    9. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    10. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    11. Sebald, Alexander, 2010. "Attribution and reciprocity," Games and Economic Behavior, Elsevier, vol. 68(1), pages 339-352, January.
    12. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    13. Geanakoplos, John & Pearce, David & Stacchetti, Ennio, 1989. "Psychological games and sequential rationality," Games and Economic Behavior, Elsevier, vol. 1(1), pages 60-79, March.
    14. Battigalli, Pierpaolo & Dufwenberg, Martin, 2009. "Dynamic psychological games," Journal of Economic Theory, Elsevier, vol. 144(1), pages 1-35, January.
    15. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
    16. Ruffle, Bradley J., 1999. "Gift giving with emotions," Journal of Economic Behavior & Organization, Elsevier, vol. 39(4), pages 399-420, July.
    17. Joshua Angrist & Eric Bettinger & Michael Kremer, 2006. "Long-Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia," American Economic Review, American Economic Association, vol. 96(3), pages 847-862, June.
    18. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    19. Zephyr, 2010. "The city," City, Taylor & Francis Journals, vol. 14(1-2), pages 154-155, February.
    20. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Returns to Capital in Microenterprises: Evidence from a Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 124(1), pages 423-423.
    21. Anup Malani, 2006. "Identifying Placebo Effects with Data from Clinical Trials," Journal of Political Economy, University of Chicago Press, vol. 114(2), pages 236-256, April.
    22. Card, David & Robins, Philip K., 2005. "How important are "entry effects" in financial incentive programs for welfare recipients? Experimental evidence from the Self-Sufficiency Project," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 113-139.
    23. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bierbrauer, Felix & Netzer, Nick, 2016. "Mechanism design and intentions," Journal of Economic Theory, Elsevier, vol. 163(C), pages 557-603.
    2. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.

    More about this item

    Keywords

    Randomized controlled trials; Policy experiments; Internal validity; Procedural concerns; Psychological game theory.;

    JEL classification:

    • C70 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D63 - Microeconomics - - Welfare Economics - - - Equity, Justice, Inequality, and Other Normative Criteria and Measurement
    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cca:wpaper:292. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Giovanni Bert). General contact details of provider: http://edirc.repec.org/data/fccaait.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.