IDEAS home Printed from https://ideas.repec.org/p/unu/wpaper/wp-2013-024.html
   My bibliography  Save this paper

Ancillary Experiments: Opportunities and Challenges

Author

Listed:
  • Kate Baldwin
  • Rikhil R. Bhavnani

Abstract

'Ancillary experiments' are a new technique whereby researchers use a completed experiment conducted by others to recover causal estimates of a randomized intervention on new outcomes. The method requires pairing new outcome data with randomized treatments the researchers themselves did not oversee. Since ancillary experiments rely on interventions that have already been undertaken, oftentimes by governments, they can provide a low-cost method with which to identify the effects of large-scale and possibly ethically difficult interventions.

Suggested Citation

  • Kate Baldwin & Rikhil R. Bhavnani, 2013. "Ancillary Experiments: Opportunities and Challenges," WIDER Working Paper Series wp-2013-024, World Institute for Development Economic Research (UNU-WIDER).
    • Handle: RePEc:unu:wpaper:wp-2013-024
    as

    Download full text from publisher

    File URL: https://www.wider.unu.edu/sites/default/files/WP2013-024.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lori Beaman & Raghabendra Chattopadhyay & Esther Duflo & Rohini Pande & Petia Topalova, 2009. "Powerful Women: Does Exposure Reduce Bias?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(4), pages 1497-1540.
    2. Jason M. Lindo & Charles Stoecker, 2014. "Drawn Into Violence: Evidence On “What Makes A Criminal” From The Vietnam Draft Lotteries," Economic Inquiry, Western Economic Association International, vol. 52(1), pages 239-258, January.
    3. Erikson, Robert S. & Stoker, Laura, 2011. "Caught in the Draft: The Effects of Vietnam Draft Lottery Status on Political Attitudes," American Political Science Review, Cambridge University Press, vol. 105(2), pages 221-237, May.
    4. Joshua D. Angrist & Alan B. Krueger, 1991. "Estimating the Payoff to Schooling Using the Vietnam-era Draft Lottery," Working Papers 670, Princeton University, Department of Economics, Industrial Relations Section..
    5. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    6. Carlos Dobkin & Reza Shabani, 2009. "The Health Effects Of Military Service: Evidence From The Vietnam Draft," Economic Inquiry, Western Economic Association International, vol. 47(1), pages 69-80, January.
    7. John Gibson & Steven Stillman & David McKenzie & Halahingano Rohorua, 2013. "Natural Experiment Evidence On The Effect Of Migration On Blood Pressure And Hypertension," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 655-672, June.
    8. David McKenzie & John Gibson & Steven Stillman, 2006. "How Important is Selection? Experimental vs Non-experimental Measures of the Income Gains of Migration," Working Papers 06_02, Motu Economic and Public Policy Research.
    9. Angrist, Joshua D. & Chen, Stacey H. & Frandsen, Brigham R., 2010. "Did Vietnam veterans get sicker in the 1990s? The complicated effects of military service on self-reported health," Journal of Public Economics, Elsevier, vol. 94(11-12), pages 824-837, December.
    10. McKenzie, David & Gibson, John & Stillman, Steven, 2006. "How important is selection ? Experimental versus non-experimental measures of the income gains from migration," Policy Research Working Paper Series 3906, The World Bank.
    11. John Gibson & David McKenzie & Steven Stillman, 2011. "The Impacts of International Migration on Remaining Household Members: Omnibus Results from a Migration Lottery Program," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1297-1318, November.
    12. Glenn W. Harrison & John A. List, 2004. "Field Experiments," Journal of Economic Literature, American Economic Association, vol. 42(4), pages 1009-1055, December.
    13. Ana L. De La O, 2013. "Do Conditional Cash Transfers Affect Electoral Behavior? Evidence from a Randomized Experiment in Mexico," American Journal of Political Science, John Wiley & Sons, vol. 57(1), pages 1-14, January.
    14. Angrist, Joshua & Chen, Stacey H., 2008. "Long-Term Economic Consequences of Vietnam-Era Conscription: Schooling, Experience and Earnings," IZA Discussion Papers 3628, Institute of Labor Economics (IZA).
    15. Manuel Bagues & Berta Esteve-Volart, 2016. "Politicians’ Luck of the Draw: Evidence from the Spanish Christmas Lottery," Journal of Political Economy, University of Chicago Press, vol. 124(5), pages 1269-1294.
    16. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    17. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    18. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    19. Glenn W. Harrison & Morten I. Lau & E. Elisabet Rutström, 2007. "Estimating Risk Attitudes in Denmark: A Field Experiment," Scandinavian Journal of Economics, Wiley Blackwell, vol. 109(2), pages 341-368, June.
    20. Sekhon, Jasjeet S. & Titiunik, Rocío, 2012. "When Natural Experiments Are Neither Natural nor Experiments," American Political Science Review, Cambridge University Press, vol. 106(1), pages 35-57, February.
    21. Bhavnani, Rikhil R., 2009. "Do Electoral Quotas Work after They Are Withdrawn? Evidence from a Natural Experiment in India," American Political Science Review, Cambridge University Press, vol. 103(1), pages 23-35, February.
    22. Michael Kremer & Dan Levy, 2008. "Peer Effects and Alcohol Use among College Students," Journal of Economic Perspectives, American Economic Association, vol. 22(3), pages 189-206, Summer.
    23. David Clingingsmith & Asim Ijaz Khwaja & Michael Kremer, 2009. "Estimating the Impact of The Hajj: Religion and Tolerance in Islam's Global Gathering," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(3), pages 1133-1170.
    24. Hastings, Justine S. & Kane, Thomas J. & Staiger, Douglas O. & Weinstein, Jeffrey M., 2007. "The effect of randomized school admissions on voter participation," Journal of Public Economics, Elsevier, vol. 91(5-6), pages 915-937, June.
    25. Sebastian Galiani & Martín A. Rossi & Ernesto Schargrodsky, 2011. "Conscription and Crime: Evidence from the Argentine Draft Lottery," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 119-136, April.
    26. Stinebrickner, Ralph & Stinebrickner, Todd R., 2006. "What can be learned about peer effects using college roommates? Evidence from new survey data and students from disadvantaged backgrounds," Journal of Public Economics, Elsevier, vol. 90(8-9), pages 1435-1454, September.
    27. Natalia Zinovyeva & Manuel F. Bagues, 2010. "Does gender matter for academic promotion? Evidence from a randomized natural experiment," Working Papers 2010-15, FEDEA.
    28. Dalton Conley & Jennifer A. Heerwig, 2009. "The Long-Term Effects of Military Conscription on Mortality: Estimates from the Vietnam-era Draft Lottery," NBER Working Papers 15105, National Bureau of Economic Research, Inc.
    29. Guido W. Imbens & Donald B. Rubin & Bruce I. Sacerdote, 2001. "Estimating the Effect of Unearned Income on Labor Earnings, Savings, and Consumption: Evidence from a Survey of Lottery Players," American Economic Review, American Economic Association, vol. 91(4), pages 778-794, September.
    30. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
    31. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    32. David Clingingsmith & Asim Ijaz Khwaja & Michael Kremer, 2009. "Estimating the Impact of The Hajj: Religion and Tolerance in Islam's Global Gathering," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(3), pages 1133-1170.
    33. Raghabendra Chattopadhyay & Esther Duflo, 2004. "Women as Policy Makers: Evidence from a Randomized Policy Experiment in India," Econometrica, Econometric Society, vol. 72(5), pages 1409-1443, September.
    34. Peter Siminski & Simon Ville, 2012. "I Was Only Nineteen, 45 Years Ago: What Can we Learn from Australia’s Conscription Lotteries?," Economics Working Papers wp12-06, School of Economics, University of Wollongong, NSW, Australia.
    35. Claudio Ferraz & Frederico Finan, 2008. "Exposing Corrupt Politicians: The Effects of Brazil's Publicly Released Audits on Electoral Outcomes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(2), pages 703-745.
    36. David McKenzie & John Gibson & Steven Stillman, 2010. "How Important Is Selection? Experimental vs. Non-Experimental Measures of the Income Gains from Migration," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 913-945, June.
    37. Stillman, Steven & McKenzie, David & Gibson, John, 2009. "Migration and mental health: Evidence from a natural experiment," Journal of Health Economics, Elsevier, vol. 28(3), pages 677-687, May.
    38. Doyle Jr., Joseph J. & Ewer, Steven M. & Wagner, Todd H., 2010. "Returns to physician human capital: Evidence from patients randomized to physician teams," Journal of Health Economics, Elsevier, vol. 29(6), pages 866-882, December.
    39. Eisenberg Daniel & Rowe Brian, 2009. "The Effect of Smoking in Young Adulthood on Smoking Later in Life: Evidence based on the Vietnam Era Draft Lottery," Forum for Health Economics & Policy, De Gruyter, vol. 12(2), pages 1-34, June.
    40. Miriam Bruhn & David McKenzie, 2009. "In Pursuit of Balance: Randomization in Practice in Development Field Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
    41. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    42. Gibson, John & McKenzie, David & Stillman, Steven, 2011. "What happens to diet and child health when migration splits households? Evidence from a migration lottery program," Food Policy, Elsevier, vol. 36(1), pages 7-15, February.
    43. Bagues, Manuel & Perez-Villadoniga, Maria J., 2012. "Do recruiters prefer applicants with similar skills? Evidence from a randomized natural experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 82(1), pages 12-20.
    44. Dunning,Thad, 2012. "Natural Experiments in the Social Sciences," Cambridge Books, Cambridge University Press, number 9781107017665, November.
    45. Kellermann, Michael & Shepsle, Kenneth A., 2009. "Congressional Careers, Committee Assignments, and Seniority Randomization in the US House of Representatives," Quarterly Journal of Political Science, now publishers, vol. 4(2), pages 87-101, July.
    46. Jonathan Guryan & Kory Kroft & Matthew J. Notowidigdo, 2009. "Peer Effects in the Workplace: Evidence from Random Groupings in Professional Golf Tournaments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 34-68, October.
    47. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    48. Rachel Milstein Sondheimer & Donald P. Green, 2010. "Using Experiments to Estimate the Effects of Education on Voter Turnout," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 174-189, January.
    49. Cecilia Elena Rouse, 1998. "Private School Vouchers and Student Achievement: An Evaluation of the Milwaukee Parental Choice Program," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 113(2), pages 553-602.
    50. Bruce Sacerdote, 2001. "Peer Effects with Random Assignment: Results for Dartmouth Roommates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 681-704.
    51. Dunning,Thad, 2012. "Natural Experiments in the Social Sciences," Cambridge Books, Cambridge University Press, number 9781107698000, November.
    52. Rogowski, Jon C. & Sinclair, Betsy, 2012. "Estimating the Causal Effects of Social Interaction with Endogenous Networks," Political Analysis, Cambridge University Press, vol. 20(3), pages 316-328, July.
    53. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    54. Yakusheva, Olga & Kapinos, Kandice & Weiss, Marianne, 2011. "Peer effects and the Freshman 15: Evidence from a natural experiment," Economics & Human Biology, Elsevier, vol. 9(2), pages 119-132, March.
    55. Green, Donald P. & Gerber, Alan S., 2002. "The Downstream Benefits of Experimentation," Political Analysis, Cambridge University Press, vol. 10(4), pages 394-402.
    56. Peter Siminski & Simon Ville, 2012. "I Was Only Nineteen, 45 Years Ago: What Can we Learn from Australia’s Conscription Lotteries? -super-,1," The Economic Record, The Economic Society of Australia, vol. 88(282), pages 351-371, September.
    57. Han, Li & Li, Tao, 2009. "The gender difference of peer influence in higher education," Economics of Education Review, Elsevier, vol. 28(1), pages 129-134, February.
    58. Sumit Agarwal & Souphala Chomsisengphet & Chunlin Liu, 2010. "The Importance of Adverse Selection in the Credit Card Market: Evidence from Randomized Trials of Credit Card Solicitations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 42(4), pages 743-754, June.
    59. Foster, Gigi, 2006. "It's not your peers, and it's not your friends: Some progress toward understanding the educational peer effect mechanism," Journal of Public Economics, Elsevier, vol. 90(8-9), pages 1455-1475, September.
    60. de Walque, Damien, 2007. "Does education affect smoking behaviors?: Evidence using the Vietnam draft as an instrument for college education," Journal of Health Economics, Elsevier, vol. 26(5), pages 877-895, September.
    61. Kosuke Imai & Dustin Tingley & Teppei Yamamoto, 2013. "Experimental designs for identifying causal mechanisms," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 5-51, January.
    62. Steven Stillman & John Gibson & David Mckenzie, 2012. "The Impact Of Immigration On Child Health: Experimental Evidence From A Migration Lottery Program," Economic Inquiry, Western Economic Association International, vol. 50(1), pages 62-81, January.
    63. Douglas H. Frank, 2012. "As Luck Would Have It: The Effect of the Vietnam Draft Lottery on Long-Term Career Outcomes," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 51(2), pages 247-274, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Baldwin, Kate & Bhavnani, Rikhil R., 2013. "Ancillary Experiments: Opportunities and Challenges," WIDER Working Paper Series 024, World Institute for Development Economic Research (UNU-WIDER).
    2. Baldwin Kate & Bhavnani Rikhil R., 2015. "Ancillary Studies of Experiments: Opportunities and Challenges," Journal of Globalization and Development, De Gruyter, vol. 6(1), pages 113-146, June.
    3. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    4. Eva Deuchert & Martin Huber, 2017. "A Cautionary Tale About Control Variables in IV Estimation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 79(3), pages 411-425, June.
    5. Jason M. Lindo & Charles Stoecker, 2014. "Drawn Into Violence: Evidence On “What Makes A Criminal” From The Vietnam Draft Lotteries," Economic Inquiry, Western Economic Association International, vol. 52(1), pages 239-258, January.
    6. Rocio Titiunik, 2020. "Natural Experiments," Papers 2002.00202, arXiv.org.
    7. Xintong Wang & Carlos A. Flores & Alfonso Flores-Lagunes, 2020. "The Effects of Vietnam-Era Military Service on the Long-Term Health of Veterans: A Bounds Analysis," Center for Policy Research Working Papers 234, Center for Policy Research, Maxwell School, Syracuse University.
    8. Sebastian Galiani & Martín A. Rossi & Ernesto Schargrodsky, 2011. "Conscription and Crime: Evidence from the Argentine Draft Lottery," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 119-136, April.
    9. Peter Siminski & Simon Ville & Alexander Paull, 2016. "Does the military turn men into criminals? New evidence from Australia’s conscription lotteries," Journal of Population Economics, Springer;European Society for Population Economics, vol. 29(1), pages 197-218, January.
    10. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    11. David McKenzie, 2012. "Learning about migration through experiments," RF Berlin - CReAM Discussion Paper Series 1207, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    12. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    13. Johnston, David W. & Shields, Michael A. & Siminski, Peter, 2016. "Long-term health effects of Vietnam-era military service: A quasi-experiment using Australian conscription lotteries," Journal of Health Economics, Elsevier, vol. 45(C), pages 12-26.
    14. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    15. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    16. Paul Bingley & Petter Lundborg & Stéphanie Vincent Lyk-Jensen, 2021. "Brothers in Arms: Spillovers from a Draft Lottery," Journal of Human Resources, University of Wisconsin Press, vol. 56(1), pages 225-268.
    17. Xintong Wang & Alfonso Flores-Lagunes, 2022. "Conscription and Military Service: Do They Result in Future Violent and Nonviolent Incarcerations and Recidivism?," Journal of Human Resources, University of Wisconsin Press, vol. 57(5), pages 1715-1757.
    18. Zhang, Hongliang, 2016. "The role of testing noise in the estimation of achievement-based peer effects," Economics of Education Review, Elsevier, vol. 54(C), pages 113-123.
    19. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    20. Alexandra de Gendre & Nicolás Salamanca, 2020. "On the Mechanisms of Ability Peer Effects," Melbourne Institute Working Paper Series wp2020n19, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:unu:wpaper:wp-2013-024. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Siméon Rapin (email available below). General contact details of provider: https://edirc.repec.org/data/widerfi.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.