IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/26158.html
   My bibliography  Save this paper

Nudging at Scale: Experimental Evidence from FAFSA Completion Campaigns

Author

Listed:
  • Kelli A. Bird
  • Benjamin L. Castleman
  • Jeffrey T. Denning
  • Joshua Goodman
  • Cait Lamberton
  • Kelly Ochs Rosinger

Abstract

Do nudge interventions that have generated positive impacts at a local level maintain efficacy when scaled state or nationwide? What specific mechanisms explain the positive impacts of promising smaller-scale nudges? We investigate, through two randomized controlled trials, the impact of a national and state-level campaign to encourage students to apply for financial aid for college. The campaigns collectively reached over 800,000 students, with multiple treatment arms to investigate different potential mechanisms. We find no impacts on financial aid receipt or college enrollment overall or for any student subgroups. We find no evidence that different approaches to message framing, delivery, or timing, or access to one-on-one advising affected campaign efficacy. We discuss why nudge strategies that work locally may be hard to scale effectively.

Suggested Citation

  • Kelli A. Bird & Benjamin L. Castleman & Jeffrey T. Denning & Joshua Goodman & Cait Lamberton & Kelly Ochs Rosinger, 2019. "Nudging at Scale: Experimental Evidence from FAFSA Completion Campaigns," NBER Working Papers 26158, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:26158
    Note: ED LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w26158.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Katharine Meyer & Kelly Ochs Rosinger, 2019. "Applying Behavioral Insights to Improve Postsecondary Education Outcomes: A Review of Obama Administration Efforts and Next Steps Under the Trump Administration," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 38(2), pages 481-499, March.
    2. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    3. Joshua Hyman, 2020. "Can Light‐Touch College‐Going Interventions Make a Difference? Evidence from a Statewide Experiment in Michigan," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(1), pages 159-190, January.
    4. Brent J. Evans & Angela Boatman & Adela Soliz, 2019. "Framing and Labeling Effects in Preferences for Borrowing for College: An Experimental Analysis," Research in Higher Education, Springer;Association for Institutional Research, vol. 60(4), pages 438-457, June.
    5. Philip Oreopoulos & Ryan Dunn, 2013. "Information and College Access: Evidence from a Randomized Field Experiment," Scandinavian Journal of Economics, Wiley Blackwell, vol. 115(1), pages 3-26, January.
    6. Taryn Dinkelman & Claudia Martínez A., 2014. "Investing in Schooling In Chile: The Role of Information about Financial Aid for Higher Education," The Review of Economics and Statistics, MIT Press, vol. 96(2), pages 244-257, May.
    7. Castleman, Benjamin L. & Page, Lindsay C., 2015. "Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates?," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 144-160.
    8. Avery, Christopher & Castleman, Benjamin L. & Hurwitz, Michael & Long, Bridget Terry & Page, Lindsay C., 2021. "Digital messaging to improve college enrollment and success," Economics of Education Review, Elsevier, vol. 84(C).
    9. Benjamin M. Marx & Lesley J. Turner, 2019. "Student Loan Nudges: Experimental Evidence on Borrowing and Educational Attainment," American Economic Journal: Economic Policy, American Economic Association, vol. 11(2), pages 108-141, May.
    10. Philip Oreopoulos & Uros Petronijevic, 2018. "Student Coaching: How Far Can Technology Go?," Journal of Human Resources, University of Wisconsin Press, vol. 53(2), pages 299-329.
    11. Oreopoulos, Philip & Petronijevic, Uros & Logel, Christine & Beattie, Graham, 2020. "Improving non-academic student outcomes using online and text-message coaching," Journal of Economic Behavior & Organization, Elsevier, vol. 171(C), pages 342-360.
    12. Booij, Adam S. & Leuven, Edwin & Oosterbeek, Hessel, 2012. "The role of information in the take-up of student loans," Economics of Education Review, Elsevier, vol. 31(1), pages 33-44.
    13. Brigitte C. Madrian & Dennis F. Shea, 2001. "The Power of Suggestion: Inertia in 401(k) Participation and Savings Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(4), pages 1149-1187.
    14. Dynarski, Susan M. & Scott–Clayton, Judith E., 2006. "The Cost of Complexity in Federal Student Aid: Lessons From Optimal Tax Theory and Behavioral Economics," National Tax Journal, National Tax Association;National Tax Journal, vol. 59(2), pages 319-356, June.
    15. Caroline M. Hoxby, 2004. "College Choices: The Economics of Where to Go, When to Go, and How to Pay For It," NBER Books, National Bureau of Economic Research, Inc, number hoxb04-1, July.
    16. Peter Bergman & Jeffrey T. Denning & Dayanand Manoli, 2019. "Is Information Enough? The Effect of Information about Education Tax Benefits on Student Outcomes," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 38(3), pages 706-731, June.
    17. Bold, Tessa & Kimenyi, Mwangi & Mwabu, Germano & Ng’ang’a, Alice & Sandefur, Justin, 2018. "Experimental evidence on scaling up education reforms in Kenya," Journal of Public Economics, Elsevier, vol. 168(C), pages 1-20.
    18. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    19. Benjamin L. Castleman & Lindsay C. Page, 2016. "Freshman Year Financial Aid Nudges: An Experiment to Increase FAFSA Renewal and College Persistence," Journal of Human Resources, University of Wisconsin Press, vol. 51(2), pages 389-415.
    20. Christopher Avery & Thomas J. Kane, 2004. "Student Perceptions of College Opportunities. The Boston COACH Program," NBER Chapters, in: College Choices: The Economics of Where to Go, When to Go, and How to Pay For It, pages 355-394, National Bureau of Economic Research, Inc.
    21. Andrew Barr & Sarah Turner, 2018. "A Letter and Encouragement: Does Information Increase Postsecondary Enrollment of UI Recipients?," American Economic Journal: Economic Policy, American Economic Association, vol. 10(3), pages 42-68, August.
    22. Caroline M. Hoxby & Sarah Turner, 2015. "What High-Achieving Low-Income Students Know about College," American Economic Review, American Economic Association, vol. 105(5), pages 514-517, May.
    23. Martin McGuigan & Sandra McNally & Gill Wyness, 2016. "Student Awareness of Costs and Benefits of Educational Decisions: Effects of an Information Campaign," Journal of Human Capital, University of Chicago Press, vol. 10(4), pages 482-519.
    24. George Bulman, 2015. "The Effect of Access to College Assessments on Enrollment and Attainment," American Economic Journal: Applied Economics, American Economic Association, vol. 7(4), pages 1-36, October.
    25. Barr, Andrew & Bird, Kelli A. & Castleman, Benjamin L., 2021. "The effect of reduced student loan borrowing on academic performance and default: Evidence from a loan counseling experiment," Journal of Public Economics, Elsevier, vol. 202(C).
    26. Oreopoulos, Philip & Petronijevic, Uros, 2019. "The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It," IZA Discussion Papers 12460, Institute of Labor Economics (IZA).
    27. Caroline Minter Hoxby, 2004. "Introduction to "College Choices: The Economics of Where to Go, When to Go, and How to Pay For It"," NBER Chapters, in: College Choices: The Economics of Where to Go, When to Go, and How to Pay For It, pages 1-12, National Bureau of Economic Research, Inc.
    28. Kelly Ochs Rosinger, 2019. "Can Simplifying Financial Aid Offers Impact College Enrollment and Borrowing? Experimental and Quasi-Experimental Evidence," Education Finance and Policy, MIT Press, vol. 14(4), pages 601-626, Fall.
    29. Benjamin Castleman & Joshua Goodman, 2018. "Intensive College Counseling and the Enrollment and Persistence of Low-Income Students," Education Finance and Policy, MIT Press, vol. 13(1), pages 19-41, Winter.
    30. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
    31. Scott Carrell & Bruce Sacerdote, 2017. "Why Do College-Going Interventions Work?," American Economic Journal: Applied Economics, American Economic Association, vol. 9(3), pages 124-151, July.
    32. Milkman, Katherine L. & Beshears, John & Choi, James J. & Laibson, David & Madrian, Brigitte C., 2012. "Following through on Good Intentions: The Power of Planning Prompts," Working Paper Series rwp12-024, Harvard University, John F. Kennedy School of Government.
    33. Amanda Pallais, 2015. "Small Differences That Matter: Mistakes in Applying to College," Journal of Labor Economics, University of Chicago Press, vol. 33(2), pages 493-520.
    34. Justin C. Ortagus & Melvin J. Tanner & Isaac McFarlin Jr., 2020. "Can Re-Enrollment Campaigns Help Dropouts Return to College? Evidence from Florida Community Colleges," NBER Working Papers 26649, National Bureau of Economic Research, Inc.
    35. French, Robert & Oreopoulos, Philip, 2017. "Behavioral barriers transitioning to college," Labour Economics, Elsevier, vol. 47(C), pages 48-63.
    36. Castleman, Benjamin L. & Owen, Laura & Page, Lindsay C., 2015. "Stay late or start early? Experimental evidence on the benefits of college matriculation support from high schools versus colleges," Economics of Education Review, Elsevier, vol. 47(C), pages 168-179.
    37. Meredith Phillips & Sarah J. Reber, 2019. "Does Virtual Advising Increase College Enrollment? Evidence from a Random Assignment College Access Field Experiment," NBER Working Papers 26509, National Bureau of Economic Research, Inc.
    38. Abhijit V. Banerjee & Esther Duflo & Rachel Glennerster, 2008. "Putting a Band-Aid on a Corpse: Incentives for Nurses in the Indian Public Health Care System," Journal of the European Economic Association, MIT Press, vol. 6(2-3), pages 487-500, 04-05.
    39. Robert Jensen, 2010. "The (Perceived) Returns to Education and the Demand for Schooling," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 515-548.
    40. Eric P. Bettinger & Bridget Terry Long & Philip Oreopoulos & Lisa Sanbonmatsu, 2012. "The Role of Application Assistance and Information in College Decisions: Results from the H&R Block Fafsa Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(3), pages 1205-1242.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    2. French, Robert & Oreopoulos, Philip, 2017. "Behavioral barriers transitioning to college," Labour Economics, Elsevier, vol. 47(C), pages 48-63.
    3. Lindsay C. Page & Judith Scott-Clayton, 2015. "Improving College Access in the United States: Barriers and Policy Responses," NBER Working Papers 21781, National Bureau of Economic Research, Inc.
    4. Page, Lindsay C. & Scott-Clayton, Judith, 2016. "Improving college access in the United States: Barriers and policy responses," Economics of Education Review, Elsevier, vol. 51(C), pages 4-22.
    5. Avery, Christopher & Castleman, Benjamin L. & Hurwitz, Michael & Long, Bridget Terry & Page, Lindsay C., 2021. "Digital messaging to improve college enrollment and success," Economics of Education Review, Elsevier, vol. 84(C).
    6. Taylor K. Odle & Jennifer A. Delaney, 2022. "You are Admitted! Early Evidence on Enrollment from Idaho’s Direct Admissions System," Research in Higher Education, Springer;Association for Institutional Research, vol. 63(6), pages 899-932, September.
    7. Bleemer, Zachary & Zafar, Basit, 2018. "Intended college attendance: Evidence from an experiment on college returns and costs," Journal of Public Economics, Elsevier, vol. 157(C), pages 184-211.
    8. Hermes, Henning & Lergetporer, Philipp & Peter, Frauke & Wiederhold, Simon, 2021. "Behavioral Barriers and the Socioeconomic Gap in Child Care Enrollment," IZA Discussion Papers 14698, Institute of Labor Economics (IZA).
    9. Henning Hermes & Philipp Lergetporer & Frauke Peter & Simon Wiederhold, 2021. "Application Barriers and the Socioeconomic Gap in Child Care Enrollment," CESifo Working Paper Series 9282, CESifo.
    10. Marx, Benjamin M. & Turner, Lesley J., 2020. "Paralysis by analysis? Effects of information on student loan take-up," Economics of Education Review, Elsevier, vol. 77(C).
    11. Adam M. Lavecchia & Heidi Liu & Philip Oreopoulos, 2014. "Behavioral Economics of Education: Progress and Possibilities," NBER Working Papers 20609, National Bureau of Economic Research, Inc.
    12. Alina Martinez & Tamara Linkow & Hannah Miller & Amanda Parsad & Cristofer Price, "undated". "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College," Mathematica Policy Research Reports c6113480c8134325ac685ddd9, Mathematica Policy Research.
    13. Oded Gurantz & Jessica Howell & Michael Hurwitz & Cassandra Larson & Matea Pender & Brooke White, 2021. "A National‐Level Informational Experiment to Promote Enrollment in Selective Colleges," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(2), pages 453-479, March.
    14. Peter, Frauke & Spiess, C. Katharina & Zambre, Vaishali, 2021. "Informing students about college: Increasing enrollment using a behavioral intervention?," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 524-549.
    15. Bonilla-Mejía, Leonardo & Bottan, Nicolas L. & Ham, Andrés, 2019. "Information policies and higher education choices experimental evidence from Colombia," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 83(C).
    16. Andrés Barrios-Fernández, 2022. "Neighbors' Effects on University Enrollment," American Economic Journal: Applied Economics, American Economic Association, vol. 14(3), pages 30-60, July.
    17. Kelly Ochs Rosinger, 2019. "Can Simplifying Financial Aid Offers Impact College Enrollment and Borrowing? Experimental and Quasi-Experimental Evidence," Education Finance and Policy, MIT Press, vol. 14(4), pages 601-626, Fall.
    18. Barnes, Stephen & Beland, Louis-Philippe & Joshi, Swarup & Willage, Barton, 2022. "Staying out of trouble? Effect of high school career counseling on crime," Economics of Education Review, Elsevier, vol. 91(C).
    19. Justin C. Ortagus & Melvin J. Tanner & Isaac McFarlin Jr., 2020. "Can Re-Enrollment Campaigns Help Dropouts Return to College? Evidence from Florida Community Colleges," NBER Working Papers 26649, National Bureau of Economic Research, Inc.
    20. de Roux, Nicolás & Riehl, Evan, 2022. "Disrupted academic careers: The returns to time off after high school," Journal of Development Economics, Elsevier, vol. 156(C).

    More about this item

    JEL classification:

    • D9 - Microeconomics - - Micro-Based Behavioral Economics
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:26158. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.