IDEAS home Printed from https://ideas.repec.org/a/inm/ormsom/v19y2017i3p347-367.html
   My bibliography  Save this article

Does Social Interaction Improve Learning Outcomes? Evidence from Field Experiments on Massive Open Online Courses

Author

Listed:
  • Dennis J. Zhang

    (Olin Business School, Washington University in St. Louis, St. Louis, Missouri 63130)

  • Gad Allon

    (Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Jan A. Van Mieghem

    (Kellogg School of Management, Northwestern University, Evanston, Illinois 60208)

Abstract

This paper studies how service providers can design social interaction among participants and quantify the causal impact of that interaction on service quality. We focus on education and analyze whether encouraging social interaction among students improves learning outcomes in massive open online courses (MOOCs), which are a new service delivery channel with universal access at reduced, if not zero, cost. We analyze three randomized experiments in a MOOC with more than 30,317 students from 183 countries. Two experiments study large-group interaction by encouraging a random subset of students to visit the course discussion board. The majority of students treated in these experiments had higher social engagement, higher quiz completion rates, and higher course grades. Using these treatments as instrumental variables, we estimate that one additional board visit causally increases the probability that a student finishes the quiz in the subsequent week by up to 4.3%. The third experiment studies small-group interaction by encouraging a random subset of students to conduct one-on-one synchronous discussions. Students who followed through and actually conducted pairwise discussions increased their quiz completion rates and quiz scores by 10% in the subsequent week. Combining results from these three experiments, we provide recommendations for designing social interaction mechanisms to improve service quality.

Suggested Citation

  • Dennis J. Zhang & Gad Allon & Jan A. Van Mieghem, 2017. "Does Social Interaction Improve Learning Outcomes? Evidence from Field Experiments on Massive Open Online Courses," Manufacturing & Service Operations Management, INFORMS, vol. 19(3), pages 347-367, July.
    • Handle: RePEc:inm:ormsom:v:19:y:2017:i:3:p:347-367
    DOI: 10.1287/msom.2016.0615
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/msom.2016.0615
    Download Restriction: no
    File URL: https://libkey.io/10.1287/msom.2016.0615?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Victor Lavy & Analía Schlosser, 2011. "Corrigendum: Mechanisms and Impacts of Gender Peer Effects at School," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 268-268, July.
    2. David J. Zimmerman, 2003. "Peer Effects in Academic Outcomes: Evidence from a Natural Experiment," The Review of Economics and Statistics, MIT Press, vol. 85(1), pages 9-23, February.
    3. Armin Falk & Andrea Ichino, 2006. "Clean Evidence on Peer Effects," Journal of Labor Economics, University of Chicago Press, vol. 24(1), pages 39-58, January.
    4. Bing Jing, 2011. "Social Learning and Dynamic Pricing of Durable Goods," Marketing Science, INFORMS, vol. 30(5), pages 851-865, September.
    5. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    6. Mary A. Burke & Tim R. Sass, 2013. "Classroom Peer Effects and Student Achievement," Journal of Labor Economics, University of Chicago Press, vol. 31(1), pages 51-82.
    7. Naor, P, 1969. "The Regulation of Queue Size by Levying Tolls," Econometrica, Econometric Society, vol. 37(1), pages 15-24, January.
    8. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    9. Yiangos Papanastasiou & Nicos Savva, 2017. "Dynamic Pricing in the Presence of Social Learning and Strategic Consumers," Management Science, INFORMS, vol. 63(4), pages 919-939, April.
    10. Zeynep Akşin & Barış Ata & Seyed Morteza Emadi & Che-Lin Su, 2013. "Structural Estimation of Callers' Delay Sensitivity in Call Centers," Management Science, INFORMS, vol. 59(12), pages 2727-2746, December.
    11. Victor Lavy & Analia Schlosser, 2011. "Mechanisms and Impacts of Gender Peer Effects at School," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 1-33, April.
    12. Gad Allon & Achal Bassamboo & Itai Gurvich, 2011. "“We Will Be Right with You”: Managing Customer Expectations with Vague Promises and Cheap Talk," Operations Research, INFORMS, vol. 59(6), pages 1382-1394, December.
    13. Bruce Sacerdote, 2001. "Peer Effects with Random Assignment: Results for Dartmouth Roommates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 681-704.
    14. Caroline Hoxby, 2000. "Peer Effects in the Classroom: Learning from Gender and Race Variation," NBER Working Papers 7867, National Bureau of Economic Research, Inc.
    15. Scott A. Imberman & Adriana D. Kugler & Bruce I. Sacerdote, 2012. "Katrina's Children: Evidence on the Structure of Peer Effects from Hurricane Evacuees," American Economic Review, American Economic Association, vol. 102(5), pages 2048-2082, August.
    16. Antoni Calvó-Armengol & Eleonora Patacchini & Yves Zenou, 2009. "Peer Effects and Social Networks in Education," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1239-1267.
    17. Senthil K. Veeraraghavan & Laurens G. Debo, 2011. "Herding in Queues with Waiting Costs: Rationality and Regret," Manufacturing & Service Operations Management, INFORMS, vol. 13(3), pages 329-346, July.
    18. David S. Lyle, 2007. "Estimating and Interpreting Peer and Role Model Effects from Randomly Assigned Social Groups at West Point," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 289-299, May.
    19. Qiuping Yu & Gad Allon & Achal Bassamboo, 2017. "How Do Delay Announcements Shape Customer Behavior? An Empirical Study," Management Science, INFORMS, vol. 63(1), pages 1-20, January.
    20. Epple, Dennis & Romano, Richard E, 1998. "Competition between Private and Public Schools, Vouchers, and Peer-Group Effects," American Economic Review, American Economic Association, vol. 88(1), pages 33-62, March.
    21. Ozan Candogan & Kostas Bimpikis & Asuman Ozdaglar, 2012. "Optimal Pricing in Networks with Externalities," Operations Research, INFORMS, vol. 60(4), pages 883-905, August.
    22. Jane Cooley Fruehwirth, 2014. "Can Achievement Peer Effect Estimates Inform Policy? A View from Inside the Black Box," The Review of Economics and Statistics, MIT Press, vol. 96(3), pages 514-523, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gong, Jie & Liu, Tracy Xiao & Tang, Jie, 2021. "How monetary incentives improve outcomes in MOOCs: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 905-921.
    2. Dennis J. Zhang & Hengchen Dai & Lingxiu Dong & Fangfang Qi & Nannan Zhang & Xiaofei Liu & Zhongyi Liu & Jiang Yang, 2020. "The Long-term and Spillover Effects of Price Promotions on Retailing Platforms: Evidence from a Large Randomized Experiment on Alibaba," Management Science, INFORMS, vol. 66(6), pages 2589-2609, June.
    3. Dennis J. Zhang & Hengchen Dai & Lingxiu Dong & Qian Wu & Lifan Guo & Xiaofei Liu, 2019. "The Value of Pop-Up Stores on Retailing Platforms: Evidence from a Field Experiment with Alibaba," Management Science, INFORMS, vol. 65(11), pages 5142-5151, November.
    4. Xiaoyan Chen & Wei Geng, 2022. "Enroll now, pay later: optimal pricing and nudge efforts for massive-online-open-courses providers," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(2), pages 1003-1018, June.
    5. Christian Terwiesch & Marcelo Olivares & Bradley R. Staats & Vishal Gaur, 2020. "OM Forum—A Review of Empirical Operations Management over the Last Two Decades," Manufacturing & Service Operations Management, INFORMS, vol. 22(4), pages 656-668, July.
    6. Andrew M. Davis & Vishal Gaur & Dayoung Kim, 2021. "Consumer Learning from Own Experience and Social Information: An Experimental Study," Management Science, INFORMS, vol. 67(5), pages 2924-2943, May.
    7. Marshall Fisher & Marcelo Olivares & Bradley R. Staats, 2020. "Why Empirical Research Is Good for Operations Management, and What Is Good Empirical Operations Management?," Manufacturing & Service Operations Management, INFORMS, vol. 22(1), pages 170-178, January.
    8. Jingchuan Pu, 2022. "Do Online Communities Improve Job Performance in the Geographically Dispersed Organization?," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 403-421, February.
    9. Pnina Feldman & Yiangos Papanastasiou & Ella Segev, 2019. "Social Learning and the Design of New Experience Goods," Management Science, INFORMS, vol. 65(5), pages 1502-1519, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gad Allon & Jan Van Mieghem & Dennis Zhang, 2016. "Does Social Interaction Improve Learning Outcomes? Field Evidence from Massive Open Online Education," Natural Field Experiments 00574, The Field Experiments Website.
    2. Alexandra de Gendre & Nicolás Salamanca, 2020. "On the Mechanisms of Ability Peer Effects," Melbourne Institute Working Paper Series wp2020n19, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    3. Silvia Mendolia & Alfredo R Paloyo & Ian Walker, 2018. "Heterogeneous effects of high school peers on educational outcomes," Oxford Economic Papers, Oxford University Press, vol. 70(3), pages 613-634.
    4. Wang, Haining & Cheng, Zhiming & Smyth, Russell, 2018. "Do migrant students affect local students’ academic achievements in urban China?," Economics of Education Review, Elsevier, vol. 63(C), pages 64-77.
    5. Michela Tincani, 2017. "Heterogeneous Peer Effects and Rank Concerns: Theory and Evidence," Working Papers 2017-006, Human Capital and Economic Opportunity Working Group.
    6. Berlinski, Samuel & Busso, Matias & Giannola, Michele, 2023. "Helping struggling students and benefiting all: Peer effects in primary education," Journal of Public Economics, Elsevier, vol. 224(C).
    7. Stephen Gibbons & Shqiponja Telhaj, 2016. "Peer Effects: Evidence from Secondary School Transition in England," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 78(4), pages 548-575, August.
    8. Zhao, Liqiu & Zhao, Zhong, 2021. "Disruptive Peers in the Classroom and Students’ Academic Outcomes: Evidence and Mechanisms," Labour Economics, Elsevier, vol. 68(C).
    9. Michela Maria Tincani, 2017. "Heterogeneous Peer Effects and Rank Concerns: Theory and Evidence," CESifo Working Paper Series 6331, CESifo.
    10. Ozkan Eren, 2016. "Differential Peer Effects, Student Achievement, and Student Absenteeism: Evidence from a Large Scale Randomized Experiment," Departmental Working Papers 2016-01, Department of Economics, Louisiana State University.
    11. Ozkan Eren, 2017. "Differential Peer Effects, Student Achievement, and Student Absenteeism: Evidence From a Large-Scale Randomized Experiment," Demography, Springer;Population Association of America (PAA), vol. 54(2), pages 745-773, April.
    12. Zhang, Hongliang, 2016. "The role of testing noise in the estimation of achievement-based peer effects," Economics of Education Review, Elsevier, vol. 54(C), pages 113-123.
    13. Clifton-Sprigg, Joanna, 2015. "Educational spillovers and parental migration," Labour Economics, Elsevier, vol. 34(C), pages 64-75.
    14. Wennberg, Karl & Norgren, Axel, 2021. "Models of Peer Effects in Education," Working Papers 21/3, Stockholm School of Economics, Center for Educational Leadership and Excellence.
    15. Li, Tao & Han, Li & Zhang, Linxiu & Rozelle, Scott, 2014. "Encouraging classroom peer interactions: Evidence from Chinese migrant schools," Journal of Public Economics, Elsevier, vol. 111(C), pages 29-45.
    16. Hill, Andrew J., 2017. "The positive influence of female college students on their male peers," Labour Economics, Elsevier, vol. 44(C), pages 151-160.
    17. Dimant, Eugen, 2019. "Contagion of pro- and anti-social behavior among peers and the role of social proximity," Journal of Economic Psychology, Elsevier, vol. 73(C), pages 66-88.
    18. Gioia De Melo, 2011. "Peer effects identified through social networks. Evidence from Uruguayan schools," Department of Economics University of Siena 627, Department of Economics, University of Siena.
    19. Feng, Han & Li, Jiayao, 2016. "Head teachers, peer effects, and student achievement," China Economic Review, Elsevier, vol. 41(C), pages 268-283.
    20. Hsieh, Chih-Sheng & Lin, Xu, 2017. "Gender and racial peer effects with endogenous network formation," Regional Science and Urban Economics, Elsevier, vol. 67(C), pages 135-147.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormsom:v:19:y:2017:i:3:p:347-367. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.