IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00681.html
   My bibliography  Save this paper

Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings

Author

Listed:
  • Bharat Chandar
  • Ali Hortacsu
  • John List
  • Ian Muir
  • Jeffrey Wooldridge

Abstract

Field experiments conducted with the village, city, state, region, or even country as the unit of randomization are becoming commonplace in the social sciences. While convenient, subsequent data analysis may be complicated by the constraint on the number of clusters in treatment and control. Through a battery of Monte Carlo simulations, we examine best practices for estimating unit-level treatment effects in cluster-randomized field experiments, particularly in settings that generate short panel data. In most settings we consider, unit-level estimation with unit fixed effects and cluster-level estimation weighted by the number of units per cluster tend to be robust to potentially problematic features in the data while giving greater statistical power. Using insights from our analysis, we evaluate the effect of a unique field experiment: a nationwide tipping field experiment across markets on the Uber app. Beyond the import of showing how tipping affects aggregate outcomes, we provide several insights on aspects of generating and analyzing cluster-randomized experimental data when there are constraints on the number of experimental units in treatment and control.

Suggested Citation

  • Bharat Chandar & Ali Hortacsu & John List & Ian Muir & Jeffrey Wooldridge, 2019. "Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings," Natural Field Experiments 00681, The Field Experiments Website.
    • Handle: RePEc:feb:natura:00681
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00681.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bharat Chandar & Uri Gneezy & John A. List & Ian Muir, 2019. "The Drivers of Social Preferences: Evidence from a Nationwide Tipping Field Experiment," NBER Working Papers 26380, National Bureau of Economic Research, Inc.
    2. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    3. Cody Cook & Rebecca Diamond & Jonathan V Hall & John A List & Paul Oyer, 2021. "The Gender Earnings Gap in the Gig Economy: Evidence from over a Million Rideshare Drivers [Measuring the Gig Economy: Current Knowledge and Open Issues]," Review of Economic Studies, Oxford University Press, vol. 88(5), pages 2210-2238.
    4. Conlin, Michael & Lynn, Michael & O'Donoghue, Ted, 2003. "The norm of restaurant tipping," Journal of Economic Behavior & Organization, Elsevier, vol. 52(3), pages 297-321, November.
    5. Susan Athey & Guido Imbens, 2016. "The Econometrics of Randomized Experiments," Papers 1607.00698, arXiv.org.
    6. Hansen, Christian B., 2007. "Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects," Journal of Econometrics, Elsevier, vol. 140(2), pages 670-694, October.
    7. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    8. Moulton, Brent R, 1990. "An Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Unit," The Review of Economics and Statistics, MIT Press, vol. 72(2), pages 334-338, May.
    9. Lynn, Michael, 2016. "Why are we more likely to tip some service occupations than others? Theory, evidence, and implications," Journal of Economic Psychology, Elsevier, vol. 54(C), pages 134-150.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jason J Sandvik & Richard E Saouma & Nathan T Seegert & Christopher T Stanton, 2020. "Workplace Knowledge Flows," The Quarterly Journal of Economics, Oxford University Press, vol. 135(3), pages 1635-1680.
    2. Basil Halperin & Benjamin Ho & John A List & Ian Muir, 2022. "Toward An Understanding of the Economics of Apologies: Evidence from a Large-Scale Natural Field Experiment," The Economic Journal, Royal Economic Society, vol. 132(641), pages 273-298.
    3. Bharat Chandar & Uri Gneezy & John A. List & Ian Muir, 2019. "The Drivers of Social Preferences: Evidence from a Nationwide Tipping Field Experiment," NBER Working Papers 26380, National Bureau of Economic Research, Inc.
    4. Cody Cook & Rebecca Diamond & Jonathan V Hall & John A List & Paul Oyer, 2021. "The Gender Earnings Gap in the Gig Economy: Evidence from over a Million Rideshare Drivers [Measuring the Gig Economy: Current Knowledge and Open Issues]," Review of Economic Studies, Oxford University Press, vol. 88(5), pages 2210-2238.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bharat Chandar & Uri Gneezy & John A. List & Ian Muir, 2019. "The Drivers of Social Preferences: Evidence from a Nationwide Tipping Field Experiment," NBER Working Papers 26380, National Bureau of Economic Research, Inc.
    2. Frank, David G. & Lynn, Michael, 2020. "Shattering the Illusion of the Self-Earned Tip: The Effect of a Restaurant Magician on Co-Workers’ Tips," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 87(C).
    3. Michael Pollmann, 2020. "Causal Inference for Spatial Treatments," Papers 2011.00373, arXiv.org, revised Jan 2023.
    4. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    5. Laurent Davezies & Xavier D'Haultfoeuille & Yannick Guyonvarch, 2018. "Asymptotic results under multiway clustering," Papers 1807.07925, arXiv.org, revised Aug 2018.
    6. Yijuan Chen & Juergen Meinecke, 2012. "Do Healthcare Report Cards Cause Providers To Select Patients And Raise Quality Of Care?," Health Economics, John Wiley & Sons, Ltd., vol. 21(S1), pages 33-55, June.
    7. Katja Görlitz & Sylvi Rzepka, 2017. "Regional training supply and employees’ training participation," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 59(1), pages 281-296, July.
    8. Chenhong Peng & Lue Fang & Julia Shu-Huah Wang & Yik Wa Law & Yi Zhang & Paul S. F. Yip, 2019. "Determinants of Poverty and Their Variation Across the Poverty Spectrum: Evidence from Hong Kong, a High-Income Society with a High Poverty Level," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 144(1), pages 219-250, July.
    9. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    10. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    11. Dick Durevall & Annika Lindskog, 2016. "Adult Mortality, AIDS, and Fertility in Rural Malawi," The Developing Economies, Institute of Developing Economies, vol. 54(3), pages 215-242, September.
    12. Bruno Ferman, 2023. "Inference in difference‐in‐differences: How much should we trust in independent clusters?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(3), pages 358-369, April.
    13. Elisabeth Christen & Michael Pfaffermayr & Yvonne Wolfmayr, 2019. "Decomposing service exports adjustments along the intensive and extensive margin at the firm‐level," Review of International Economics, Wiley Blackwell, vol. 27(1), pages 155-183, February.
    14. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2018. "Effects of Market Structure and Patient Choice on Hospital Quality for Planned Patients," School of Economics Discussion Papers 1118, School of Economics, University of Surrey.
    15. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    16. Brewer, Mike & Crossley, Thomas F. & Zilio, Federico, 2019. "What Do We Really Know about the Employment Effects of the UK's National Minimum Wage?," IZA Discussion Papers 12369, Institute of Labor Economics (IZA).
    17. Elsa Perdrix & Quitterie Roquebert, 2020. "Does an increase in formal care affect informal care ? Evidence among the French elderly," Working Papers halshs-02370689, HAL.
    18. Pedro Pita Barros, 2022. "Quality decreases from introducing patient choice in a National Health Service," Portuguese Economic Journal, Springer;Instituto Superior de Economia e Gestao, vol. 21(3), pages 351-381, September.
    19. Kunze, Lars & Suppa, Nicolai, 2017. "Bowling alone or bowling at all? The effect of unemployment on social participation," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 213-235.
    20. Elif Aydin, Asli & Acun, Yüksel, 2019. "An investigation of tipping behavior as a major component in service economy: The case of taxi tipping," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 78(C), pages 114-120.

    More about this item

    JEL classification:

    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • C5 - Mathematical and Quantitative Methods - - Econometric Modeling
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00681. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.fieldexperiments.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: David Franks (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.