IDEAS home Printed from https://ideas.repec.org/a/nat/nature/v600y2021i7889d10.1038_s41586-021-04128-4.html
   My bibliography  Save this article

Megastudies improve the impact of applied behavioural science

Author

Listed:
  • Katherine L. Milkman

    (University of Pennsylvania)

  • Dena Gromet

    (University of Pennsylvania)

  • Hung Ho

    (University of Pennsylvania
    University of Chicago)

  • Joseph S. Kay

    (University of Pennsylvania)

  • Timothy W. Lee

    (University of Pennsylvania
    Northwestern University)

  • Pepi Pandiloski

    (University of Chicago)

  • Yeji Park

    (Princeton University)

  • Aneesh Rai

    (University of Pennsylvania)

  • Max Bazerman

    (Harvard University)

  • John Beshears

    (Harvard University)

  • Lauri Bonacorsi

    (Northwestern University)

  • Colin Camerer

    (California Institute of Technology)

  • Edward Chang

    (Harvard University)

  • Gretchen Chapman

    (Carnegie Mellon University)

  • Robert Cialdini

    (Arizona State University)

  • Hengchen Dai

    (University of California Los Angeles)

  • Lauren Eskreis-Winkler

    (University of Chicago)

  • Ayelet Fishbach

    (University of Chicago)

  • James J. Gross

    (Stanford University)

  • Samantha Horn

    (Carnegie Mellon University)

  • Alexa Hubbard

    (New York University)

  • Steven J. Jones

    (Rutgers University)

  • Dean Karlan

    (Northwestern University)

  • Tim Kautz

    (Mathematica)

  • Erika Kirgios

    (University of Pennsylvania)

  • Joowon Klusowski

    (University of Pennsylvania)

  • Ariella Kristal

    (Harvard University)

  • Rahul Ladhania

    (University of Michigan)

  • George Loewenstein

    (Carnegie Mellon University)

  • Jens Ludwig

    (University of Chicago)

  • Barbara Mellers

    (University of Pennsylvania)

  • Sendhil Mullainathan

    (University of Chicago)

  • Silvia Saccardo

    (Carnegie Mellon University)

  • Jann Spiess

    (Stanford Graduate School of Business)

  • Gaurav Suri

    (San Francisco State University)

  • Joachim H. Talloen

    (Carnegie Mellon University)

  • Jamie Taxer

    (Stanford University)

  • Yaacov Trope

    (New York University)

  • Lyle Ungar

    (University of Pennsylvania)

  • Kevin G. Volpp

    (University of Pennsylvania)

  • Ashley Whillans

    (Harvard University)

  • Jonathan Zinman

    (Dartmouth College)

  • Angela L. Duckworth

    (University of Pennsylvania
    University of Pennsylvania)

Abstract

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes1. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals2. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research3–6. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

Suggested Citation

  • Katherine L. Milkman & Dena Gromet & Hung Ho & Joseph S. Kay & Timothy W. Lee & Pepi Pandiloski & Yeji Park & Aneesh Rai & Max Bazerman & John Beshears & Lauri Bonacorsi & Colin Camerer & Edward Chang, 2021. "Megastudies improve the impact of applied behavioural science," Nature, Nature, vol. 600(7889), pages 478-483, December.
  • Handle: RePEc:nat:nature:v:600:y:2021:i:7889:d:10.1038_s41586-021-04128-4
    DOI: 10.1038/s41586-021-04128-4
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41586-021-04128-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1038/s41586-021-04128-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Erev, Ido & Hiller, Maximilian & Klößner, Stefan & Lifshitz, Gal & Mertins, Vanessa & Roth, Yefim, 2022. "Promoting healthy behavior through repeated deposit contracts: An intervention study," Journal of Economic Psychology, Elsevier, vol. 92(C).
    2. Grace McKeon & Chiara Mastrogiovanni & Megan Teychenne & Simon Rosenbaum, 2022. "Barriers and Facilitators to Participating in an Exercise Referral Scheme among Women Living in a Low Socioeconomic Area in Australia: A Qualitative Investigation Using the COM-B and Theoretical Domai," IJERPH, MDPI, vol. 19(19), pages 1-13, September.
    3. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    4. Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023. "Heterogeneity in effect size estimates: Empirical evidence and practical implications," Working Papers 2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
    5. Linus Dahlander, 2022. "The role of autonomy and selection at the gate in flat organizations," Journal of Organization Design, Springer;Organizational Design Community, vol. 11(1), pages 27-29, March.
    6. Eugen Dimant & Shaul Shalvi, 2022. "Meta-Nudging Honesty: Past, Present, and Future of the Research Frontier," ECONtribute Discussion Papers Series 163, University of Bonn and University of Cologne, Germany.
    7. Timmons, Shane & Robertson, Deirdre & Lunn, Pete, 2022. "Combining nudges and boosts to increase precautionary saving: A large-scale field experiment," Papers WP722, Economic and Social Research Institute (ESRI).
    8. Anna Gaysynsky & Kathryn Heley & Wen-Ying Sylvia Chou, 2022. "An Overview of Innovative Approaches to Support Timely and Agile Health Communication Research and Practice," IJERPH, MDPI, vol. 19(22), pages 1-24, November.
    9. Voelkel, Jan G. & Stagnaro, Michael & Chu, James & Pink, Sophia Lerner & Mernyk, Joseph S. & Redekopp, Chrystal & Ghezae, Isaias & Cashman, Matthew & Adjodah, Dhaval & Allen, Levi, 2023. "Megastudy identifying effective interventions to strengthen Americans’ democratic attitudes," OSF Preprints y79u5, Center for Open Science.
    10. Lewańczyk, Agata Marta & Langham-Walsh, Eleanor & Edwards, Lisa & Branney, Peter & Walters, Elizabeth R. & Mitchell, Paul & Vaportzis, Eleftheria, 2023. "Back Onside protocol: A physical activity intervention to improve health outcomes in people who are unemployed or at risk of unemployment," Evaluation and Program Planning, Elsevier, vol. 97(C).
    11. Diane Pelly & Orla Doyle, 2022. "Nudging in the workplace: increasing participation in employee EDI wellness events," Working Papers 202208, Geary Institute, University College Dublin.
    12. Dillon Bowen, 2022. "Simple models predict behavior at least as well as behavioral scientists," Papers 2208.01167, arXiv.org.
    13. Arad, Ayala & Gneezy, Uri & Mograbi, Eli, 2023. "Intermittent incentives to encourage exercising in the long run," Journal of Economic Behavior & Organization, Elsevier, vol. 205(C), pages 560-573.
    14. Polman, Evan & Ruttan, Rachel L. & Peck, Joann, 2022. "Using curiosity to incentivize the choice of “should” options," Organizational Behavior and Human Decision Processes, Elsevier, vol. 173(C).
    15. Andrej Woerner, 2021. "Overcoming Time Inconsistency with a Matched Bet: Theory and Evidence from Exercising," CESifo Working Paper Series 9503, CESifo.
    16. Emile Bruneau & Andrés Casas & Boaz Hameiri & Nour Kteily, 2022. "Exposure to a media intervention helps promote support for peace in Colombia," Nature Human Behaviour, Nature, vol. 6(6), pages 847-857, June.
    17. Woerner, Andrej, 2023. "Overcoming Time Inconsistency with a Matched Bet: Theory and Evidence from Exercising," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277711, Verein für Socialpolitik / German Economic Association.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nature:v:600:y:2021:i:7889:d:10.1038_s41586-021-04128-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.