IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00681.html
   My bibliography  Save this paper

Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings

Author

Listed:
  • Bharat Chandar
  • Ali Hortacsu
  • John List
  • Ian Muir
  • Jeffrey Wooldridge

Abstract

Field experiments conducted with the village, city, state, region, or even country as the unit of randomization are becoming commonplace in the social sciences. While convenient, subsequent data analysis may be complicated by the constraint on the number of clusters in treatment and control. Through a battery of Monte Carlo simulations, we examine best practices for estimating unit-level treatment effects in cluster-randomized field experiments, particularly in settings that generate short panel data. In most settings we consider, unit-level estimation with unit fixed effects and cluster-level estimation weighted by the number of units per cluster tend to be robust to potentially problematic features in the data while giving greater statistical power. Using insights from our analysis, we evaluate the effect of a unique field experiment: a nationwide tipping field experiment across markets on the Uber app. Beyond the import of showing how tipping affects aggregate outcomes, we provide several insights on aspects of generating and analyzing cluster-randomized experimental data when there are constraints on the number of experimental units in treatment and control.

Suggested Citation

  • Bharat Chandar & Ali Hortacsu & John List & Ian Muir & Jeffrey Wooldridge, 2019. "Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings," Natural Field Experiments 00681, The Field Experiments Website.
    • Handle: RePEc:feb:natura:00681
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00681.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Conlin, Michael & Lynn, Michael & O'Donoghue, Ted, 2003. "The norm of restaurant tipping," Journal of Economic Behavior & Organization, Elsevier, vol. 52(3), pages 297-321, November.
    2. Hansen, Christian B., 2007. "Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects," Journal of Econometrics, Elsevier, vol. 140(2), pages 670-694, October.
    3. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    4. Cody Cook & Rebecca Diamond & Jonathan Hall & John A. List & Paul Oyer, 2018. "The Gender Earnings Gap in the Gig Economy: Evidence from over a Million Rideshare Drivers," NBER Working Papers 24732, National Bureau of Economic Research, Inc.
    5. Lynn, Michael, 2016. "Why are we more likely to tip some service occupations than others? Theory, evidence, and implications," Journal of Economic Psychology, Elsevier, vol. 54(C), pages 134-150.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bharat Chandar & Uri Gneezy & John List & Ian Muir, 2019. "The Drivers of Social Preferences: Evidence from a Nationwide Tipping Field Experiment," Natural Field Experiments 00680, The Field Experiments Website.
    2. Basil Halperin & Benjamin Ho & John List & Ian Muir, 2018. "Toward an understanding of the economics of apologies: evidence from a large-scale natural field experiment," Natural Field Experiments 00644, The Field Experiments Website.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bharat Chandar & Uri Gneezy & John A. List & Ian Muir, 2019. "The Drivers of Social Preferences: Evidence from a Nationwide Tipping Field Experiment," NBER Working Papers 26380, National Bureau of Economic Research, Inc.
    2. Elif Aydin, Asli & Acun, Yüksel, 2019. "An investigation of tipping behavior as a major component in service economy: The case of taxi tipping," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 78(C), pages 114-120.
    3. Nicolas Moreau, 2018. "A SAS macro for estimation and inference in differences-in-differences applications with only a few treated groups," Working Papers hal-01691486, HAL.
    4. Frank, David G. & Lynn, Michael, 2020. "Shattering the Illusion of the Self-Earned Tip: The Effect of a Restaurant Magician on Co-Workers’ Tips," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 87(C).
    5. Nicolas Moreau, 2018. "A SAS macro for estimation and inference in differences-in-differences applications with within cluster correlation and cluster-corrections for a few clusters," Working Papers hal-01691476, HAL.
    6. Michael Pollmann, 2020. "Causal Inference for Spatial Treatments," Papers 2011.00373, arXiv.org.
    7. Jongmoo Jay Choi & Hoje Jo & Jimi Kim & Moo Sung Kim, 2018. "Business Groups and Corporate Social Responsibility," Journal of Business Ethics, Springer, vol. 153(4), pages 931-954, December.
    8. Francesca Carta & Lucia Rizzica, 2015. "Female employment and pre-kindergarten: on the uninteded effects of an Italian reform," Temi di discussione (Economic working papers) 1030, Bank of Italy, Economic Research and International Relations Area.
    9. Rotunno, Lorenzo, 2016. "Political stability and trade agreements: Evidence for ‘endgame FTAs’," European Journal of Political Economy, Elsevier, vol. 45(C), pages 133-148.
    10. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    11. Nora Gordon & Sarah Reber, 2018. "The effects of school desegregation on mixed-race births," Journal of Population Economics, Springer;European Society for Population Economics, vol. 31(2), pages 561-596, April.
    12. Berthélemy Michel & Bonev Petyo & Dussaux Damien & Söderberg Magnus, 2019. "Methods for strengthening a weak instrument in the case of a persistent treatment," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 23(1), pages 1-30, February.
    13. Erin L. Krupka & Roberto A. Weber, 2013. "Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary?," Journal of the European Economic Association, European Economic Association, vol. 11(3), pages 495-524, June.
    14. Hollingsworth, Bruce & Ohinata, Asako & Picchio, Matteo & Walker, Ian, 2017. "Labour supply and informal care supply: The impacts of financial support for long-term elderly care," GLO Discussion Paper Series 118, Global Labor Organization (GLO).
    15. Guido Friebel & Matthias Heinz & Miriam Krueger & Nikolay Zubanov, 2017. "Team Incentives and Performance: Evidence from a Retail Chain," American Economic Review, American Economic Association, vol. 107(8), pages 2168-2203, August.
    16. Silva,Joana C. G. & Morgandi,Matteo & Levin,Victoria, 2016. "Trust in government and support for redistribution," Policy Research Working Paper Series 7675, The World Bank.
    17. Carrieri, Vincenzo & Madio, Leonardo & Principe, Francesco, 2019. "Light cannabis and organized crime: Evidence from (unintended) liberalization in Italy," European Economic Review, Elsevier, vol. 113(C), pages 63-76.
    18. Ronelle Burger & Canh Thien Dang & Trudy Owens, 2017. "Better performing NGOs do report more accurately: Evidence from investigating Ugandan NGO financial accounts," Discussion Papers 2017-10, University of Nottingham, CREDIT.
    19. Andrés Elberg & Pedro M. Gardete & Rosario Macera & Carlos Noton, 2019. "Dynamic effects of price promotions: field evidence, consumer search, and supply-side implications," Quantitative Marketing and Economics (QME), Springer, vol. 17(1), pages 1-58, March.
    20. Cain Polidano & Justin van de Ven & Sarah Voitchovsky, 2017. "The Power of Self-Interest: Effects of Education and Training Entitlements in Later-Life," Melbourne Institute Working Paper Series wp2017n12, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.

    More about this item

    JEL classification:

    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • C5 - Mathematical and Quantitative Methods - - Econometric Modeling
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00681. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Joe Seidel). General contact details of provider: http://www.fieldexperiments.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.