IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00756.html
   My bibliography  Save this paper

Returns to effort: experimental evidence from an online language platform

Author

Listed:
  • Fulya Ersoy

Abstract

While distance learning has become widespread, causal estimates regarding returns to effort in technology-assisted learning environments are scarce due to high attrition rates and endogeneity of effort. In this paper, I manipulate effort by randomly assigning students different numbers of lessons in a popular online language learning platform. Using administrative data from the platform and the instrumental variables strategy, I find that completing 9 Duolingo lessons, which corresponds to approximately 60 minutes of studying, leads to a 0.057-0.095 standard deviation increase in test scores. Comparisons to the literature and back-of-the-envelope calculations suggest that distance learning can be as effective as in-person learning for college students for an introductory language course

Suggested Citation

  • Fulya Ersoy, 2021. "Returns to effort: experimental evidence from an online language platform," Natural Field Experiments 00756, The Field Experiments Website.
  • Handle: RePEc:feb:natura:00756
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00756.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, Oxford University Press, vol. 122(3), pages 1235-1264.
    2. Roland G. Fryer, 2011. "Financial Incentives and Student Achievement: Evidence from Randomized Trials," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1755-1798.
    3. Dobkin, Carlos & Gil, Ricard & Marion, Justin, 2010. "Skipping class in college and exam performance: Evidence from a regression discontinuity classroom experiment," Economics of Education Review, Elsevier, vol. 29(4), pages 566-575, August.
    4. Stinebrickner Ralph & Stinebrickner Todd R., 2008. "The Causal Effect of Studying on Academic Performance," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(1), pages 1-55, June.
    5. Papia Bawa, 2016. "Retention in Online Courses," SAGE Open, , vol. 6(1), pages 21582440156, January.
    6. Damon Clark & David Gill & Victoria Prowse & Mark Rush, 2020. "Using Goals to Motivate College Students: Theory and Evidence From Field Experiments," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 648-663, October.
    7. Lisa Barrow & Lisa Markman & Cecilia Elena Rouse, 2009. "Technology's Edge: The Educational Benefits of Computer-Aided Instruction," American Economic Journal: Economic Policy, American Economic Association, vol. 1(1), pages 52-74, February.
    8. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    9. Joshua Goodman & Julia Melkers & Amanda Pallais, 2019. "Can Online Delivery Increase Access to Education?," Journal of Labor Economics, University of Chicago Press, vol. 37(1), pages 1-34.
    10. Lai, Fang & Luo, Renfu & Zhang, Linxiu & Huang, Xinzhe & Rozelle, Scott, 2015. "Does computer-assisted learning improve learning outcomes? Evidence from a randomized experiment in migrant schools in Beijing," Economics of Education Review, Elsevier, vol. 47(C), pages 34-48.
    11. Eric P. Bettinger & Lindsay Fox & Susanna Loeb & Eric S. Taylor, 2017. "Virtual Classrooms: How Online College Courses Affect Student Success," American Economic Review, American Economic Association, vol. 107(9), pages 2855-2875, September.
    12. Eren, Ozkan & Henderson, Daniel J., 2011. "Are we wasting our children's time by giving them more homework?," Economics of Education Review, Elsevier, vol. 30(5), pages 950-961, October.
    13. Emir Kamenica, 2012. "Behavioral Economics and Psychology of Incentives," Annual Review of Economics, Annual Reviews, vol. 4(1), pages 427-452, July.
    14. Dalia Ghanem & Sarojini Hirshleifer & Karen Ortiz-Becerra, 2019. "Testing for Attrition Bias in Field Experiments," Working Papers 202010, University of California at Riverside, Department of Economics, revised Mar 2020.
    15. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    16. Ghanem, Dalia & Hirshleifer, Sarojini & Ortiz-Becerra, Karen, 2019. "Testing Attrition Bias in Field Experiments," 2019 Annual Meeting, July 21-23, Atlanta, Georgia 291215, Agricultural and Applied Economics Association.
    17. Glewwe, P. & Muralidharan, K., 2016. "Improving Education Outcomes in Developing Countries," Handbook of the Economics of Education,, Elsevier.
    18. Ozkan Eren & Daniel J. Henderson, 2008. "The impact of homework on student achievement," Econometrics Journal, Royal Economic Society, vol. 11(2), pages 326-348, July.
    19. Kuehn, Zoe & Landeras, Pedro, 2012. "Study Time and Scholarly Achievement in PISA," MPRA Paper 49033, University Library of Munich, Germany.
    20. David J. Deming & Claudia Goldin & Lawrence F. Katz & Noam Yuchtman, 2015. "Can Online Learning Bend the Higher Education Cost Curve?," American Economic Review, American Economic Association, vol. 105(5), pages 496-501, May.
    21. Gregory A. Krohn & Catherine M. O'Connor, 2005. "Student Effort and Performance over the Semester," The Journal of Economic Education, Taylor & Francis Journals, vol. 36(1), pages 3-28, January.
    22. Metcalfe, Robert & Burgess, Simon & Proud, Steven, 2019. "Students' effort and educational achievement: Using the timing of the World Cup to vary the value of leisure," Journal of Public Economics, Elsevier, vol. 172(C), pages 111-126.
    23. Arnaud Chevalier & Peter Dolton & Melanie Lührmann, 2018. "‘Making it count’: incentives, student effort and performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(2), pages 323-349, February.
    24. Joshua Angrist & Victor Lavy, 2009. "The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial," American Economic Review, American Economic Association, vol. 99(4), pages 1384-1414, September.
    25. Davidson, Russell & MacKinnon, James G., 2010. "Wild Bootstrap Tests for IV Regression," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 128-144.
    26. Hans Bonesrønning & Leiv Opstad, 2015. "Can student effort be manipulated? Does it matter?," Applied Economics, Taylor & Francis Journals, vol. 47(15), pages 1511-1524, March.
    27. Gianni De Fraja & Tania Oliveira & Luisa Zanchi, 2010. "Must Try Harder: Evaluating the Role of Effort in Educational Attainment," The Review of Economics and Statistics, MIT Press, vol. 92(3), pages 577-597, August.
    28. Stinebrickner, Ralph & Stinebrickner, T.R.Todd R., 2004. "Time-use and college outcomes," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 243-269.
    29. Sarojini Hirshleifer, 2017. "Incentives for Effort or Outputs? A Field Experiment to Improve Student Performance," Working Papers 201701, University of California at Riverside, Department of Economics.
    30. Philip Oreopoulos & Richard W. Patterson & Uros Petronijevic & Nolan G. Pope, 2018. "When Studying and Nudging Don’t Go as Planned: Unsuccessful Attempts to Help Traditional and Online College Students," NBER Working Papers 25036, National Bureau of Economic Research, Inc.
    31. Jennjou Chen & Tsui-Fang Lin, 2008. "Class Attendance and Exam Performance: A Randomized Experiment," The Journal of Economic Education, Taylor & Francis Journals, vol. 39(3), pages 213-227, July.
    32. Andrew Grodner & Nicholas G. Rupp, 2013. "The Role of Homework in Student Learning Outcomes: Evidence from a Field Experiment," The Journal of Economic Education, Taylor & Francis Journals, vol. 44(2), pages 93-109, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ersoy, Fulya, 2023. "Effects of perceived productivity on study effort: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 207(C), pages 376-391.
    2. Akcay, Erol & Ohashi, Ryotaro, 2023. "The floating duck syndrome: biased social learning leads to effort-reward imbalances," SocArXiv qx7ku, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rosa Sanchis-Guarner & José Montalbán & Felix Weinhardt, 2021. "Home Broadband and Human Capital Formation," CESifo Working Paper Series 8846, CESifo.
    2. Chadi, Adrian & de Pinto, Marco & Schultze, Gabriel, 2019. "Young, gifted and lazy? The role of ability and labor market prospects in student effort decisions," Economics of Education Review, Elsevier, vol. 72(C), pages 66-79.
    3. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    4. Islam, Asad & Kwon, Sungoh & Masood, Eema & Prakash, Nishith & Sabarwal, Shwetlena & Saraswat, Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," GLO Discussion Paper Series 526, Global Labor Organization (GLO).
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Arulampalam, Wiji & Naylor, Robin A. & Smith, Jeremy, 2012. "Am I missing something? The effects of absence from class on student performance," Economics of Education Review, Elsevier, vol. 31(4), pages 363-375.
    7. Bin Tang & Te-Tien Ting & Chyi-In Wu & Yue Ma & Di Mo & Wei-Ting Hung & Scott Rozelle, 2020. "The Impact of Online Computer Assisted Learning at Home for Disadvantaged Children in Taiwan: Evidence from a Randomized Experiment," Sustainability, MDPI, vol. 12(23), pages 1-16, December.
    8. Ersoy, Fulya, 2023. "Effects of perceived productivity on study effort: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 207(C), pages 376-391.
    9. Allen, James & Mahumane, Arlete & Riddell, James & Rosenblat, Tanya & Yang, Dean & Yu, Hang, 2022. "Teaching and incentives: Substitutes or complements?," Economics of Education Review, Elsevier, vol. 91(C).
    10. Gong, Jie & Liu, Tracy Xiao & Tang, Jie, 2021. "How monetary incentives improve outcomes in MOOCs: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 905-921.
    11. Yi Lu & Hong Song, 2020. "The effect of educational technology on college students’ labor market performance," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(3), pages 1101-1126, July.
    12. Lisa Barrow & Cecilia Elena Rouse, 2018. "Financial Incentives and Educational Investment: The Impact of Performance-based Scholarships on Student Time Use," Education Finance and Policy, MIT Press, vol. 13(4), pages 419-448, Fall.
    13. James Berry, 2015. "Child Control in Education Decisions: An Evaluation of Targeted Incentives to Learn in India," Journal of Human Resources, University of Wisconsin Press, vol. 50(4), pages 1051-1080.
    14. Nerea Gómez-Fernández & Mauro Mediavilla, 2018. "Do information and communication technologies (ICT) improve educational outcomes? Evidence for Spain in PISA 2015," Working Papers 2018/20, Institut d'Economia de Barcelona (IEB).
    15. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    16. Cardim, Joana & Molina-Millán, Teresa & Vicente, Pedro C., 2023. "Can technology improve the classroom experience in primary education? An African experiment on a worldwide program," Journal of Development Economics, Elsevier, vol. 164(C).
    17. Torberg Falch & Marte Rønning, 2011. "Homework assignment and student achievement in OECD countries," Working Paper Series 11411, Department of Economics, Norwegian University of Science and Technology.
    18. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    19. Bach, Maximilian & Fischer, Mira, 2020. "Understanding the response to high-stakes incentives in primary education," ZEW Discussion Papers 20-066, ZEW - Leibniz Centre for European Economic Research.
    20. Robert Metcalfe & Simon Burgess and Steven Proud, 2011. "Student effort and educational attainment: Using the England football team to identify the education production function," Economics Series Working Papers 586, University of Oxford, Department of Economics.

    More about this item

    JEL classification:

    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I26 - Health, Education, and Welfare - - Education - - - Returns to Education
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00756. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: David Franks (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.