IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v6y2022i11d10.1038_s41562-022-01437-0.html
   My bibliography  Save this article

A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries

Author

Listed:
  • Katherine L. Milkman

    (University of Pennsylvania)

  • Linnea Gandhi

    (University of Pennsylvania)

  • Sean F. Ellis

    (University of Pennsylvania)

  • Heather N. Graci

    (University of Pennsylvania)

  • Dena M. Gromet

    (University of Pennsylvania)

  • Rayyan S. Mobarak

    (University of Pennsylvania)

  • Alison M. Buttenheim

    (University of Pennsylvania
    University of Pennsylvania)

  • Angela L. Duckworth

    (University of Pennsylvania
    University of Pennsylvania)

  • Devin Pope

    (University of Chicago)

  • Ala Stanford

    (US Department of Health and Human Services)

  • Richard Thaler

    (University of Chicago)

  • Kevin G. Volpp

    (University of Pennsylvania
    University of Pennsylvania)

Abstract

Lotteries have been shown to motivate behaviour change in many settings, but their value as a policy tool is relatively untested. We implemented a pre-registered, citywide experiment to test the effects of three high-pay-off, geographically targeted lotteries designed to motivate adult Philadelphians to get their COVID-19 vaccine. In each drawing, the residents of a randomly selected ‘treatment’ zip code received half the lottery prizes, boosting their chances of winning to 50×–100× those of other Philadelphians. The first treated zip code, which drew considerable media attention, may have experienced a small bump in vaccinations compared with the control zip codes: average weekly vaccinations rose by an estimated 61 per 100,000 people per week (+11%). After pooling the results from all three zip codes treated during our six-week experiment, however, we do not detect evidence of any overall benefits. Furthermore, our 95% confidence interval provides a 9% upper bound on the net benefits of treatment in our study.

Suggested Citation

  • Katherine L. Milkman & Linnea Gandhi & Sean F. Ellis & Heather N. Graci & Dena M. Gromet & Rayyan S. Mobarak & Alison M. Buttenheim & Angela L. Duckworth & Devin Pope & Ala Stanford & Richard Thaler &, 2022. "A citywide experiment testing the impact of geographically targeted, high-pay-off vaccine lotteries," Nature Human Behaviour, Nature, vol. 6(11), pages 1515-1524, November.
    • Handle: RePEc:nat:nathum:v:6:y:2022:i:11:d:10.1038_s41562-022-01437-0
    DOI: 10.1038/s41562-022-01437-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-022-01437-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1038/s41562-022-01437-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. Nicola Lacetera & Devin G. Pope & Justin R. Sydnor, 2012. "Heuristic Thinking and Limited Attention in the Car Market," American Economic Review, American Economic Association, vol. 102(5), pages 2206-2236, August.
    4. Sebastian Galiani & Brian Quistorff, 2017. "The synth runner package: Utilities to automate synthetic control estimation using synth," Stata Journal, StataCorp LLC, vol. 17(4), pages 834-849, December.
    5. Eduardo Cavallo & Sebastian Galiani & Ilan Noy & Juan Pantano, 2013. "Catastrophic Natural Disasters and Economic Growth," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1549-1561, December.
    6. Margaret E. Brehm & Paul A. Brehm & Martin Saavedra, 2022. "The Ohio Vaccine Lottery and Starting Vaccination Rates," American Journal of Health Economics, University of Chicago Press, vol. 8(3), pages 387-411.
    7. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    8. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    9. MacKinnon, James G. & Webb, Matthew D., 2018. "Randomization Inference for Difference-in-Differences with Few Treated Clusters," Queen's Economics Department Working Papers 274681, Queen's University - Department of Economics.
    10. Arindrajit Dube & Ethan Kaplan & Suresh Naidu, 2011. "Coups, Corporations, and Classified Information," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(3), pages 1375-1409.
    11. Drazen Prelec, 1998. "The Probability Weighting Function," Econometrica, Econometric Society, vol. 66(3), pages 497-528, May.
    12. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    13. Hengchen Dai & Silvia Saccardo & Maria A. Han & Lily Roh & Naveen Raja & Sitaram Vangala & Hardikkumar Modi & Shital Pandya & Michael Sloyan & Daniel M. Croymans, 2021. "Behavioural nudges increase COVID-19 vaccinations," Nature, Nature, vol. 597(7876), pages 404-409, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elinder, Mikael & Erixson, Oscar & Öhman, Mattias, 2023. "Cognitive ability, health policy, and the dynamics of COVID-19 vaccination," Journal of Health Economics, Elsevier, vol. 91(C).
    2. Upravitelev, A., 2023. "Efficacious methods of restraining COVID-19 through behavioral public policy," Journal of the New Economic Association, New Economic Association, vol. 61(4), pages 203-222.
    3. Pol Campos-Mercade & Armando N. Meier & Stephan Meier & Devin G. Pope & Florian H. Schneider & Erik Wengström, 2024. "Incentives to Vaccinate," NBER Working Papers 32899, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Webb, Matthew D., 2014. "Reworking Wild Bootstrap Based Inference for Clustered Errors," Queen's Economics Department Working Papers 274640, Queen's University - Department of Economics.
    2. Matthew D. Webb, 2023. "Reworking wild bootstrap‐based inference for clustered errors," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(3), pages 839-858, August.
    3. repec:osf:osfxxx:s8ayp_v1 is not listed on IDEAS
    4. Peter Backus & Thien Nguyen, 2021. "The Effect of the Sex Buyer Law on the Market for Sex, Sexual Health and Sexual Violence," Economics Discussion Paper Series 2106, Economics, The University of Manchester.
    5. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    6. Denti, Daria & Iammarino, Simona, 2022. "Coming Out of the Woods. Do local support services influence the propensity to report sexual violence?," Journal of Economic Behavior & Organization, Elsevier, vol. 193(C), pages 334-352.
    7. Bruce E. Hansen, 2025. "Standard Errors for Difference‐in‐Difference Regression," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 40(3), pages 291-309, April.
    8. Lindo, Jason M. & Pineda-Torres, Mayra, 2021. "New Evidence on the Effects of Mandatory Waiting Periods for Abortion," Journal of Health Economics, Elsevier, vol. 80(C).
    9. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    10. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    11. Pablo Javier Garofalo & Jorge M. Streb, 2022. "Broken Promises: Regime Announcements and Exchange Rates around Elections," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 1-32.
    12. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    13. Arturo Antón & Alejandro Rasteletti, 2022. "Taxing Labor Income in an Economy with High Employment Informality," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 33-68.
    14. Bruno Ferman & Cristine Pinto, 2019. "Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 452-467, July.
    15. Antonio David & Takuji Komatsuzaki & Samuel Pienknagura, 2022. "The Macroeconomic and Socioeconomic Effects of Structural Reforms in Latin America and the Caribbean," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 115-155.
    16. Joseph, Iverson-Love, 2022. "The effect of natural disaster on economic growth: Evidence from a major earthquake in Haiti," World Development, Elsevier, vol. 159(C).
    17. Magdalena Blanco & Jose Maria Cabrera & Felipe Carozzi & Alejandro Cid de Orta, 2022. "Mandatory Helmet Use and the Severity of Motorcycle Accidents: No Brainer?," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 187-218, June.
    18. Rodríguez-Puello, Gabriel, 2024. "Digging for Trouble? Uncovering the Link Between Mining Booms and Crime," OSF Preprints s8ayp, Center for Open Science.
    19. Ali Mohammad Alichi & Ippei Shibata & Kadir Tanyeri, 2022. "Fiscal Policy Multipliers in Small States," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 69-114.
    20. Christopher O'Leary & Tulio Cravo & Ana Cristina Sierra & Leandro Justino, 2022. "Effects of Job Referrals on Labor Market Outcomes in Brazil," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 157-186.
    21. Blanco, Magdalena & Cabrera, José María & Cid, Alejandro & Carozzi, Felipe, 2017. "Effects of Motorcycle Helmet Laws on Fatalities’ Prevention: An Impact Evaluation," MPRA Paper 84652, University Library of Munich, Germany.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:6:y:2022:i:11:d:10.1038_s41562-022-01437-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.