IDEAS home Printed from https://ideas.repec.org/a/oup/wbrobs/v33y2018i1p34-64..html
   My bibliography  Save this article

Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity

Author

Listed:
  • Jörg Peters
  • Jörg Langbein
  • Gareth Roberts

Abstract

When properly implemented, Randomized Controlled Trials (RCT) achieve a high degree of internal validity. Yet, if an RCT is to inform policy, it is critical to establish external validity. This paper systematically reviews all RCTs conducted in developing countries and published in leading economic journals between 2009 and 2014 with respect to how they deal with external validity. Following Duflo, Glennerster, and Kremer (2008), we scrutinize the following hazards to external validity: Hawthorne effects, general equilibrium effects, specific sample problems, and special care in treatment provision. Based on a set of objective indicators, we find that the majority of published RCTs does not discuss these hazards and many do not provide the necessary information to assess potential problems. The paper calls for including external validity dimensions in a more systematic reporting on the results of RCTs. This may create incentives to avoid overgeneralizing findings and help policy makers to interpret results appropriately.

Suggested Citation

  • Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," World Bank Research Observer, World Bank Group, vol. 33(1), pages 34-64.
  • Handle: RePEc:oup:wbrobs:v:33:y:2018:i:1:p:34-64.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/wbro/lkx005
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lori Beaman & Raghabendra Chattopadhyay & Esther Duflo & Rohini Pande & Petia Topalova, 2009. "Powerful Women: Does Exposure Reduce Bias?," The Quarterly Journal of Economics, Oxford University Press, vol. 124(4), pages 1497-1540.
    2. Pedro C. Vicente, 2014. "Is Vote Buying Effective? Evidence from a Field Experiment in West Africa," Economic Journal, Royal Economic Society, vol. 124(574), pages 356-387, February.
    3. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 531-580.
    4. Esther Duflo & Michael Kremer & Jonathan Robinson, 2011. "Nudging Farmers to Use Fertilizer: Theory and Experimental Evidence from Kenya," American Economic Review, American Economic Association, vol. 101(6), pages 2350-2390, October.
    5. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Are Women More Credit Constrained? Experimental Evidence on Gender and Microenterprise Returns," American Economic Journal: Applied Economics, American Economic Association, vol. 1(3), pages 1-32, July.
    6. Jobiba Chinkhumba & Susan Godlonton & Rebecca Thornton, 2014. "The Demand for Medical Male Circumcision," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 152-177, April.
    7. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    8. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
    9. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    10. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," World Bank Economic Review, World Bank Group, vol. 31(3), pages 687-707.
    11. Dean Karlan & Martin Valdivia, 2011. "Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 510-527, May.
    12. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    13. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    14. Lori Beaman & Jeremy Magruder, 2012. "Who Gets the Job Referral? Evidence from a Social Networks Experiment," American Economic Review, American Economic Association, vol. 102(7), pages 3574-3593, December.
    15. Uri Gneezy & Kenneth L. Leonard & John A. List, 2009. "Gender Differences in Competition: Evidence From a Matrilineal and a Patriarchal Society," Econometrica, Econometric Society, vol. 77(5), pages 1637-1664, September.
    16. Nicholas Bloom & Benn Eifert & Aprajit Mahajan & David McKenzie & John Roberts, 2013. "Does Management Matter? Evidence from India," The Quarterly Journal of Economics, Oxford University Press, vol. 128(1), pages 1-51.
    17. Benjamin Feigenberg & Erica Field & Rohini Pande, 2013. "The Economic Returns to Social Interaction: Experimental Evidence from Microfinance," Review of Economic Studies, Oxford University Press, vol. 80(4), pages 1459-1483.
    18. Elizabeth M. King & Jere R. Behrman, 2009. "Timing and Duration of Exposure in Evaluations of Social Programs," World Bank Research Observer, World Bank Group, vol. 24(1), pages 55-82, February.
    19. Paul Collier & Pedro C. Vicente, 2014. "Votes and Violence: Evidence from a Field Experiment in Nigeria," Economic Journal, Royal Economic Society, vol. 124(574), pages 327-355, February.
    20. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    21. Nava Ashraf, 2009. "Spousal Control and Intra-household Decision Making: An Experimental Study in the Philippines," American Economic Review, American Economic Association, vol. 99(4), pages 1245-1277, September.
    22. Esther Duflo & Rema Hanna & Stephen P. Ryan, 2012. "Incentives Work: Getting Teachers to Come to School," American Economic Review, American Economic Association, vol. 102(4), pages 1241-1278, June.
    23. Dean Karlan & Robert Osei & Isaac Osei-Akoto & Christopher Udry, 2014. "Agricultural Decisions after Relaxing Credit and Risk Constraints," The Quarterly Journal of Economics, Oxford University Press, vol. 129(2), pages 597-652.
    24. Michael Kremer & Edward Miguel & Rebecca Thornton, 2009. "Incentives to Learn," The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 437-456, August.
    25. Karen Macours & Renos Vakis, 2014. "Changing Households' Investment Behaviour through Social Interactions with Local Leaders: Evidence from a Randomised Transfer Programme," Economic Journal, Royal Economic Society, vol. 0(576), pages 607-633, May.
    26. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, Oxford University Press, vol. 130(3), pages 1117-1165.
    27. Alejandro Drexler & Greg Fischer & Antoinette Schoar, 2014. "Keeping It Simple: Financial Literacy and Rules of Thumb," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 1-31, April.
    28. Katherine Casey & Rachel Glennerster & Edward Miguel, 2012. "Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan," The Quarterly Journal of Economics, Oxford University Press, vol. 127(4), pages 1755-1812.
    29. Jenny C. Aker & Christopher Ksoll & Travis J. Lybbert, 2012. "Can Mobile Phones Improve Learning? Evidence from a Field Experiment in Niger," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 94-120, October.
    30. David K. Evans & Anna Popova, 2016. "What Really Works to Improve Learning in Developing Countries? An Analysis of Divergent Findings in Systematic Reviews," World Bank Research Observer, World Bank Group, vol. 31(2), pages 242-270.
    31. Michael Woolcock, 2013. "Using Case Studies to Explore the External Validity of ‘Complex’ Development Interventions," CID Working Papers 270, Center for International Development at Harvard University.
    32. Esther Duflo & Michael Greenstone & Nicholas Ryan, 2013. "Truth-telling by Third-party Auditors and the Response of Polluting Firms: Experimental Evidence from India," The Quarterly Journal of Economics, Oxford University Press, vol. 128(4), pages 1499-1545.
    33. Paul Glewwe & Nauman Ilias & Michael Kremer, 2010. "Teacher Incentives," American Economic Journal: Applied Economics, American Economic Association, vol. 2(3), pages 205-227, July.
    34. Orazio Attanasio & Abigail Barr & Juan Camilo Cardenas & Garance Genicot & Costas Meghir, 2012. "Risk Pooling, Risk Preferences, and Social Networks," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 134-167, April.
    35. Vivi Alatas & Abhijit Banerjee & Rema Hanna & Benjamin A. Olken & Julia Tobias, 2012. "Targeting the Poor: Evidence from a Field Experiment in Indonesia," American Economic Review, American Economic Association, vol. 102(4), pages 1206-1240, June.
    36. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 263-306.
    37. Martina Björkman & Jakob Svensson, 2009. "Power to the People: Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda," The Quarterly Journal of Economics, Oxford University Press, vol. 124(2), pages 735-769.
    38. Suresh de Mel & David McKenzie & Christopher Woodruff, 2013. "The Demand for, and Consequences of, Formalization among Informal Firms in Sri Lanka," American Economic Journal: Applied Economics, American Economic Association, vol. 5(2), pages 122-150, April.
    39. Xavier Giné & Dean Karlan & Jonathan Zinman, 2010. "Put Your Money Where Your Butt Is: A Commitment Contract for Smoking Cessation," American Economic Journal: Applied Economics, American Economic Association, vol. 2(4), pages 213-235, October.
    40. David P. Coady & Rebecca L. Harris, 2004. "Evaluating transfer programmes within a general equilibrium framework," Economic Journal, Royal Economic Society, vol. 114(498), pages 778-799, October.
    41. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    42. Nava Ashraf & Erica Field & Jean Lee, 2014. "Household Bargaining and Excess Fertility: An Experimental Study in Zambia," American Economic Review, American Economic Association, vol. 104(7), pages 2210-2237, July.
    43. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    44. Michal Bauer & Julie Chytilova & Jonathan Morduch, 2012. "Behavioral Foundations of Microcredit: Experimental and Survey Evidence from Rural India," American Economic Review, American Economic Association, vol. 102(2), pages 1118-1139, April.
    45. Sylvie Moulin & Michael Kremer & Paul Glewwe, 2009. "Many Children Left Behind? Textbooks and Test Scores in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 112-135, January.
    46. repec:oup:qjecon:v:129:y:2014:i:2:p:697-752. is not listed on IDEAS
    47. Pascaline Dupas, 2014. "Short‐Run Subsidies and Long‐Run Adoption of New Health Products: Evidence From a Field Experiment," Econometrica, Econometric Society, vol. 82(1), pages 197-228, January.
    48. Jonathan Robinson, 2012. "Limited Insurance within the Household: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 140-164, October.
    49. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    50. Jonas Hjort, 2013. "Ethnic Divisions and Production in Firms," CESifo Working Paper Series 4449, CESifo.
    51. David McKenzie & Christopher Woodruff, 2014. "What Are We Learning from Business Training and Entrepreneurship Evaluations around the Developing World?," World Bank Research Observer, World Bank Group, vol. 29(1), pages 48-82.
    52. Pascaline Dupas & Jonathan Robinson, 2013. "Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 163-192, January.
    53. Dean Karlan & Jonathan Zinman, 2009. "Observing Unobservables: Identifying Information Asymmetries With a Consumer Credit Field Experiment," Econometrica, Econometric Society, vol. 77(6), pages 1993-2008, November.
    54. Pascaline Dupas & Jonathan Robinson, 2013. "Why Don't the Poor Save More? Evidence from Health Savings Experiments," American Economic Review, American Economic Association, vol. 103(4), pages 1138-1171, June.
    55. Gani Aldashev & Georg Kirchsteiger & Alexander Sebald, 2017. "Assignment Procedure Biases in Randomised Policy Experiments," Economic Journal, Royal Economic Society, vol. 127(602), pages 873-895, June.
    56. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    57. Xavier Gine & Jessica Goldberg & Dean Yang, 2012. "Credit Market Consequences of Improved Personal Identification: Field Experimental Evidence from Malawi," American Economic Review, American Economic Association, vol. 102(6), pages 2923-2954, October.
    58. Felipe Barrera-Osorio & Marianne Bertrand & Leigh L. Linden & Francisco Perez-Calle, 2011. "Improving the Design of Conditional Transfer Programs: Evidence from a Randomized Education Experiment in Colombia," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 167-195, April.
    59. David K. Evans & Anna Popova, 2017. "Cash Transfers and Temptation Goods," Economic Development and Cultural Change, University of Chicago Press, vol. 65(2), pages 189-221.
    60. Jishnu Das & Stefan Dercon & James Habyarimana & Pramila Krishnan & Karthik Muralidharan & Venkatesh Sundararaman, 2013. "School Inputs, Household Substitution, and Test Scores," American Economic Journal: Applied Economics, American Economic Association, vol. 5(2), pages 29-57, April.
    61. Brian E. Roe & David R. Just, 2009. "Internal and External Validity in Economics Research: Tradeoffs between Experiments, Field Experiments, Natural Experiments, and Field Data," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 91(5), pages 1266-1271.
    62. Sarah Baird & Craig McIntosh & Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1709-1753.
    63. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
    64. Erica Field & Rohini Pande & John Papp & Natalia Rigol, 2013. "Does the Classic Microfinance Model Discourage Entrepreneurship among the Poor? Experimental Evidence from India," American Economic Review, American Economic Association, vol. 103(6), pages 2196-2226, October.
    65. Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    66. Orazio Attanasio & Adriana Kugler & Costas Meghir, 2011. "Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 188-220, July.
    67. Dana Burde & Leigh L. Linden, 2013. "Bringing Education to Afghan Girls: A Randomized Controlled Trial of Village-Based Schools," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 27-40, July.
    68. repec:idb:brikps:77778 is not listed on IDEAS
    69. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    70. Timothy J. Besley & Konrad B. Burchardi & Maitreesh Ghatak, 2012. "Incentives and the De Soto Effect," The Quarterly Journal of Economics, Oxford University Press, vol. 127(1), pages 237-282.
    71. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    72. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    73. Robert Jensen, 2012. "Do Labor Market Opportunities Affect Young Women's Work and Family Decisions? Experimental Evidence from India," The Quarterly Journal of Economics, Oxford University Press, vol. 127(2), pages 753-792.
    74. Christopher Blattman & Nathan Fiala & Sebastian Martinez, 2014. "Generating Skilled Self-Employment in Developing Countries: Experimental Evidence from Uganda," The Quarterly Journal of Economics, Oxford University Press, vol. 129(2), pages 697-752.
    75. Karthik Muralidharan & Venkatesh Sundararaman, 2010. "The Impact of Diagnostic Feedback to Teachers on Student Learning: Experimental Evidence from India," Economic Journal, Royal Economic Society, vol. 120(546), pages 187-203, August.
    76. repec:idb:brikps:publication-detail,7101.html?id=32886 is not listed on IDEAS
    77. Moussa P. Blimpo, 2014. "Team Incentives for Education in Developing Countries: A Randomized Field Experiment in Benin," American Economic Journal: Applied Economics, American Economic Association, vol. 6(4), pages 90-109, October.
    78. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
    79. Menno Pradhan & Daniel Suryadarma & Amanda Beatty & Maisy Wong & Arya Gaduh & Armida Alisjahbana & Rima Prama Artha, 2014. "Improving Educational Quality through Enhancing Community Participation: Results from a Randomized Field Experiment in Indonesia," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 105-126, April.
    80. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    81. repec:feb:artefa:0087 is not listed on IDEAS
    82. Seán M. Muller, 2015. "Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations," World Bank Economic Review, World Bank Group, vol. 29(suppl_1), pages 217-225.
    83. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    84. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    85. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 1-45.
    86. Karthik Muralidharan & Venkatesh Sundararaman, 2011. "Teacher Performance Pay: Experimental Evidence from India," Journal of Political Economy, University of Chicago Press, vol. 119(1), pages 39-77.
    87. Karthik Muralidharan & Paul Niehaus & Sandip Sukhtankar, 2017. "General Equilibrium Effects of (Improving) Public Employment Programs: Experimental Evidence from India," NBER Working Papers 23838, National Bureau of Economic Research, Inc.
    88. Alessandro Tarozzi & Aprajit Mahajan & Brian Blackburn & Dan Kopf & Lakshmi Krishnan & Joanne Yoong, 2014. "Micro-loans, Insecticide-Treated Bednets, and Malaria: Evidence from a Randomized Controlled Trial in Orissa, India," American Economic Review, American Economic Association, vol. 104(7), pages 1909-1941, July.
    89. Rema Hanna & Sendhil Mullainathan & Joshua Schwartzstein, 2014. "Learning Through Noticing: Theory and Evidence from a Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 129(3), pages 1311-1353.
    90. Kaushik Basu, 2014. "Randomisation, Causality and the Role of Reasoned Intuition," Oxford Development Studies, Taylor & Francis Journals, vol. 42(4), pages 455-472, December.
    91. Hongbin Cai & Yuyu Chen & Hanming Fang, 2009. "Observational Learning: Evidence from a Randomized Natural Field Experiment," American Economic Review, American Economic Association, vol. 99(3), pages 864-882, June.
    92. Karthik Muralidharan & Venkatesh Sundararaman, 2015. "Editor's Choice The Aggregate Effect of School Choice: Evidence from a Two-Stage Experiment in India," The Quarterly Journal of Economics, Oxford University Press, vol. 130(3), pages 1011-1066.
    93. Maarten J. Voors & Eleonora E. M. Nillesen & Philip Verwimp & Erwin H. Bulte & Robert Lensink & Daan P. Van Soest, 2012. "Violent Conflict and Behavior: A Field Experiment in Burundi," American Economic Review, American Economic Association, vol. 102(2), pages 941-964, April.
    94. Amanda E. Kowalski, 2018. "How to Examine External Validity Within an Experiment," NBER Working Papers 24834, National Bureau of Economic Research, Inc.
    95. Paul J. Gertler & Sebastian W. Martinez & Marta Rubio-Codina, 2012. "Investing Cash Transfers to Raise Long-Term Living Standards," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 164-192, January.
    96. Leonardo Bursztyn & Lucas C. Coffman, 2012. "The Schooling Decision: Family Preferences, Intergenerational Conflict, and Moral Hazard in the Brazilian Favelas," Journal of Political Economy, University of Chicago Press, vol. 120(3), pages 359-397.
    97. Benjamin A. Olken & Junko Onishi & Susan Wong, 2014. "Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia," American Economic Journal: Applied Economics, American Economic Association, vol. 6(4), pages 1-34, October.
    98. Lant Pritchett & Justin Sandefur, 2015. "Learning from Experiments When Context Matters," American Economic Review, American Economic Association, vol. 105(5), pages 471-475, May.
    99. Olivier Armantier & Amadou Boly, 2013. "Comparing Corruption in the Laboratory and in the Field in Burkina Faso and in Canada," Economic Journal, Royal Economic Society, vol. 123(12), pages 1168-1187, December.
    100. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Returns to Capital in Microenterprises: Evidence from a Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 124(1), pages 423-423.
    101. Emily Oster & Rebecca Thornton, 2011. "Menstruation, Sanitary Products, and School Attendance: Evidence from a Randomized Evaluation," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 91-100, January.
    102. Robert T. Jensen & Nolan H. Miller, 2011. "Do Consumer Price Subsidies Really Improve Nutrition?," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1205-1223, November.
    103. Robert Jensen, 2010. "The (Perceived) Returns to Education and the Demand for Schooling," The Quarterly Journal of Economics, Oxford University Press, vol. 125(2), pages 515-548.
    104. Thomas Fujiwara & Leonard Wantchekon, 2013. "Can Informed Public Deliberation Overcome Clientelism? Experimental Evidence from Benin," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 241-255, October.
    105. Cilliers, Jacobus & Dube, Oeindrila & Siddiqi, Bilal, 2015. "The white-man effect: How foreigner presence affects behavior in experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 118(C), pages 397-414.
    106. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    107. Erwin Bulte & Gonne Beekman & Salvatore Di Falco & Joseph Hella & Pan Lei, 2014. "Behavioral Responses and the Impact of New Agricultural Technologies: Evidence from a Double-blind Field Experiment in Tanzania," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 96(3), pages 813-830.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fiala, Nathan & Neubauer, Florian & Peters, Jörg, 2022. "Do economists replicate?," Ruhr Economic Papers 939, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    2. Andrew Dustan & Juan Manuel Hernandez-Agramonte & Stanislao Maldonado, 2018. "Motivating bureaucrats with non-monetary incentives when state capacity is weak: Evidence from large-scale," Natural Field Experiments 00664, The Field Experiments Website.
    3. Gunther Bensch & Jörg Peters, 2020. "One‐Off Subsidies and Long‐Run Adoption—Experimental Evidence on Improved Cooking Stoves in Senegal," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(1), pages 72-90, January.
    4. Pablo Brañas-Garza & Lorenzo Estepa-Mohedano & Diego Jorrat & Victor Orozco & Ericka Rascón-Ramírez, 2021. "To pay or not to pay: Measuring risk preferences in lab and field," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 16(5), pages 1290-1313, September.
    5. Andor, Mark A. & Fels, Katja M. & Renz, Jan & Rzepka, Sylvi, 2018. "Do planning prompts increase educational success? Evidence from randomized controlled trials in MOOCs," Ruhr Economic Papers 790, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Dustan, Andrew & Maldonado, Stanislao & Hernandez-Agramonte, Juan Manuel, 2018. "Motivating bureaucrats with non-monetary incentives when state capacity is weak: Evidence from large-scale field experiments in Peru," MPRA Paper 90952, University Library of Munich, Germany.
    7. Ali, Saleem H., 2020. "Environmental urgency versus the allure of RCT empiricism," World Development, Elsevier, vol. 127(C).
    8. MacNeill Timothy & Vibert Amber, 2019. "Universal Basic Income and the Natural Environment: Theory and Policy," Basic Income Studies, De Gruyter, vol. 14(1), pages 1-15, June.
    9. Kvangraven, Ingrid Harvold, 2020. "Impoverished economics? A critical assessment of the new gold standard," World Development, Elsevier, vol. 127(C).
    10. Andor, Mark A. & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2020. "Social Norms and Energy Conservation Beyond the US," Journal of Environmental Economics and Management, Elsevier, vol. 103(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    2. Teresa Molina Millan & Karen Macours, 2017. "Attrition in randomized control trials: Using tracking information to correct bias," NOVAFRICA Working Paper Series wp1702, Universidade Nova de Lisboa, Nova School of Business and Economics, NOVAFRICA.
    3. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    4. Benjamin A. Olken, 2020. "Banerjee, Duflo, Kremer, and the Rise of Modern Development Economics," Scandinavian Journal of Economics, Wiley Blackwell, vol. 122(3), pages 853-878, July.
    5. Pascaline Dupas & Edward Miguel, 2016. "Impacts and Determinants of Health Levels in Low-Income Countries," NBER Working Papers 22235, National Bureau of Economic Research, Inc.
    6. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    7. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    8. Harounan Kazianga & Leigh Linden & Ali Protik & Matt Sloan, 2015. "Impact Evaluation of Burkina Faso's BRIGHT Program: Design Report," Mathematica Policy Research Reports c0250cd3f27d448ea70d909c3, Mathematica Policy Research.
    9. Simone Schaner, 2018. "The Persistent Power of Behavioral Change: Long-Run Impacts of Temporary Savings Subsidies for the Poor," American Economic Journal: Applied Economics, American Economic Association, vol. 10(3), pages 67-100, July.
    10. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    11. Dean Karlan & Aishwarya Lakshmi Ratan & Jonathan Zinman, 2014. "Savings by and for the Poor: A Research Review and Agenda," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 60(1), pages 36-78, March.
    12. Eric Bettinger & Nina Cunha & Guilherme Lichand & Ricardo Madeira, 2020. "Are the effects of informational interventions driven by salience?," ECON - Working Papers 350, Department of Economics - University of Zurich, revised May 2021.
    13. Francesca Marchetta & Tom Dilly, 2019. "Supporting Education in Africa: Opportunities and Challenges for an Impact Investor," Working Papers hal-02288103, HAL.
    14. Teresa Molina Millán & Karen Macours, 2017. "Attrition in randomized control trials: Using tracking information to correct bias," FEUNL Working Paper Series novaf:wp1702, Universidade Nova de Lisboa, Faculdade de Economia.
    15. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    16. Paulina Oliva & B. Kelsey Jack & Samuel Bell & Elizabeth Mettetal & Christopher Severen, 2020. "Technology Adoption under Uncertainty: Take-Up and Subsequent Investment in Zambia," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 617-632, July.
    17. Meredith, Jennifer & Robinson, Jonathan & Walker, Sarah & Wydick, Bruce, 2013. "Keeping the doctor away: Experimental evidence on investment in preventative health products," Journal of Development Economics, Elsevier, vol. 105(C), pages 196-210.
    18. Clair Null & Clemencia Cosentino & Swetha Sridharan & Laura Meyer, "undated". "Policies and Programs to Improve Secondary Education in Developing Countries: A Review of the Evidence," Mathematica Policy Research Reports 516e420e637c4851b15e6a3f6, Mathematica Policy Research.
    19. Abhijit V. Banerjee & Esther Duflo, 2010. "Giving Credit Where It Is Due," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 61-80, Summer.
    20. Grant Miller & A. Mushfiq Mobarak, 2013. "Gender Differences in Preferences, Intra-Household Externalities, and Low Demand for Improved Cookstoves," NBER Working Papers 18964, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:wbrobs:v:33:y:2018:i:1:p:34-64.. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/wrldbus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press or Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/wrldbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.