IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/24004.html
   My bibliography  Save this paper

Measuring Success in Education: The Role of Effort on the Test Itself

Author

Listed:
  • Uri Gneezy
  • John A. List
  • Jeffrey A. Livingston
  • Sally Sadoff
  • Xiangdong Qin
  • Yang Xu

Abstract

Tests measuring and comparing educational achievement are an important policy tool. We experimentally show that offering students extrinsic incentives to put forth effort on such achievement tests has differential effects across cultures. Offering incentives to U.S. students, who generally perform poorly on assessments, improved performance substantially. In contrast, Shanghai students, who are top performers on assessments, were not affected by incentives. Our findings suggest that in the absence of extrinsic incentives, ranking countries based on low-stakes assessments is problematic because test scores reflect differences in intrinsic motivation to perform well on the test itself, and not just differences in ability.

Suggested Citation

  • Uri Gneezy & John A. List & Jeffrey A. Livingston & Sally Sadoff & Xiangdong Qin & Yang Xu, 2017. "Measuring Success in Education: The Role of Effort on the Test Itself," NBER Working Papers 24004, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:24004
    Note: ED
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w24004.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Eric A. Hanushek & Ludger Woessmann, 2010. "How much do Educational Outcomes Matter in OECD Countries?," CESifo Working Paper Series 3238, CESifo.
    2. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    3. Uri Gneezy & Stephan Meier & Pedro Rey-Biel, 2011. "When and Why Incentives (Don't) Work to Modify Behavior," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 191-210, Fall.
    4. Jalava, Nina & Joensen, Juanna Schrøter & Pellas, Elin, 2015. "Grades and rank: Impacts of non-financial incentives on test performance," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 161-196.
    5. Steven D. Levitt & John A. List & Susanne Neckermann & Sally Sadoff, 2016. "The Behavioralist Goes to School: Leveraging Behavioral Economics to Improve Educational Performance," American Economic Journal: Economic Policy, American Economic Association, vol. 8(4), pages 183-219, November.
    6. Ludger Woessmann, 2016. "The Importance of School Systems: Evidence from International Differences in Student Achievement," CESifo Working Paper Series 5951, CESifo.
    7. Eric A. Hanushek & Ludger Woessmann, 2011. "How much do educational outcomes matter in OECD countries? [‘Accountability and flexibility in public schools: Evidence from Boston’s charters and pilots’]," Economic Policy, CEPR;CES;MSH, vol. 26(67), pages 427-491.
    8. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    9. Carmit Segal, 2012. "Working When No One Is Watching: Motivation, Test Scores, and Economic Success," Management Science, INFORMS, vol. 58(8), pages 1438-1457, August.
    10. Ghazala Azmat & Caterina Calsamiglia & Nagore Iriberri, 2016. "Gender Differences In Response To Big Stakes," Journal of the European Economic Association, European Economic Association, vol. 14(6), pages 1372-1400, December.
    11. Ludger Woessmann, 2016. "The Importance of School Systems: Evidence from International Differences in Student Achievement," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 3-32, Summer.
    12. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    13. Jacob, Brian A., 2005. "Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 761-796, June.
    14. Stefano DellaVigna & Devin Pope, 2018. "What Motivates Effort? Evidence and Expert Forecasts," Review of Economic Studies, Oxford University Press, vol. 85(2), pages 1029-1069.
    15. Attali, Yigal & Neeman, Zvika & Schlosser, Analia, 2011. "Rise to the Challenge or Not Give a Damn: Differential Performance in High vs. Low Stakes Tests," IZA Discussion Papers 5693, Institute of Labor Economics (IZA).
    16. Wößmann, Ludger, 2016. "The Importance of School Systems," Munich Reprints in Economics 43463, University of Munich, Department of Economics.
    17. Imas, Alex, 2014. "Working for the “warm glow”: On the benefits and limits of prosocial incentives," Journal of Public Economics, Elsevier, vol. 114(C), pages 14-18.
    18. Richard J. Murnane & John B. Willett & Yves Duhaldeborde & John H. Tyler, 2000. "How important are the cognitive skills of teenagers in predicting subsequent earnings?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 19(4), pages 547-568.
    19. Balart, Pau & Oosterveen, Matthijs & Webbink, Dinand, 2015. "Test Scores, Noncognitive Skills and Economic Growth," IZA Discussion Papers 9559, Institute of Labor Economics (IZA).
    20. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    21. Attali, Yigal & Neeman, Zvika & Schlosser, Analia, 2011. "Rise to the Challenge or Not Give a Damn: Differential Performance in High vs. Low Stakes Tests," Foerder Institute for Economic Research Working Papers 275743, Tel-Aviv University > Foerder Institute for Economic Research.
    22. Schmid, Friedrich & Trede, Mark, 1995. "A distribution free test for the two sample problem for general alternatives," Computational Statistics & Data Analysis, Elsevier, vol. 20(4), pages 409-419, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Measuring success in education: the role of effort on the test itself
      by maximorossi in NEP-LTV blog on 2018-03-20 13:38:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Francesca Borgonovi & Alessandro Ferrara & Mario Piacentini, 2020. "From asking to observing. Behavioural measures of socio-emotional and motivational skills in large-scale assessments," DoQSS Working Papers 20-19, Quantitative Social Science - UCL Social Research Institute, University College London.
    2. List, John A. & Samek, Anya & Suskind, Dana L., 2018. "Combining behavioral economics and field experiments to reimagine early childhood education," Behavioural Public Policy, Cambridge University Press, vol. 2(1), pages 1-21, May.
    3. Matthias Stefan & Jürgen Huber & Michael Kirchler & Matthias Sutter & Markus Walzl, 2020. "Monetary and Social Incentives in Multi-Tasking: The Ranking Substitution Effect," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2020_10, Max Planck Institute for Research on Collective Goods.
    4. Lucas Gortazar, 2019. "¿Favorece el sistema educativo español la igualdad de oportunidades?," Studies on the Spanish Economy eee2019-17, FEDEA.
    5. Isaac Mbiti & Karthik Muralidharan & Mauricio Romero & Youdi Schipper & Constantine Manda & Rakesh Rajani, 2019. "Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania," The Quarterly Journal of Economics, Oxford University Press, vol. 134(3), pages 1627-1673.
    6. Yong Cai, 2021. "A Modified Randomization Test for the Level of Clustering," Papers 2105.01008, arXiv.org.
    7. Iriberri, Nagore & Rey-Biel, Pedro, 2021. "Brave boys and play-it-safe girls: Gender differences in willingness to guess in a large scale natural field experiment," European Economic Review, Elsevier, vol. 131(C).
    8. Yong Cai, 2021. "Some Finite Sample Properties of the Sign Test," Papers 2103.01412, arXiv.org.
    9. Filmer, Deon & Rogers, Halsey & Angrist, Noam & Sabarwal, Shwetlena, 2020. "Learning-adjusted years of schooling (LAYS): Defining a new macro measure of education," Economics of Education Review, Elsevier, vol. 77(C).
    10. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2020. "School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic," NBER Working Papers 28205, National Bureau of Economic Research, Inc.
    11. Bernal, Gloria L. & Penney, Jeffrey, 2019. "Scholarships and student effort: Evidence from Colombia’s Ser Pilo Paga program," Economics of Education Review, Elsevier, vol. 72(C), pages 121-130.
    12. Charness, Gary & List, John A. & Rustichini, Aldo & Samek, Anya & Van De Ven, Jeroen, 2019. "Theory of mind among disadvantaged children: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 174-194.
    13. List, John A. & Livingston, Jeffrey A. & Neckermann, Susanne, 2018. "Do financial incentives crowd out intrinsic motivation to perform on standardized tests?," Economics of Education Review, Elsevier, vol. 66(C), pages 125-136.
    14. Brunello, Giorgio & Crema, Angela & Rocco, Lorenzo, 2018. "Testing at Length If It Is Cognitive or Non-Cognitive," IZA Discussion Papers 11603, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiqian Cai & Yi Lu & Jessica Pan & Songfa Zhong, 2019. "Gender Gap under Pressure: Evidence from China's National College Entrance Examination," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 249-263, May.
    2. Jalava, Nina & Joensen, Juanna Schrøter & Pellas, Elin, 2015. "Grades and rank: Impacts of non-financial incentives on test performance," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 161-196.
    3. Islam,Asad & Kwon,Sungoh & Masood,Eema & Prakash,Nishith & Sabarwal,Shwetlena & Saraswat,Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," Policy Research Working Paper Series 9406, The World Bank.
    4. Gomez-Gonzalez, Jose Eduardo & Rodríguez Gómez, Wilson & Rodríguez Gómez, Efrén, 2021. "Explaining the Rural-Urban Student Performance Gap for Different Distribution Quantiles in Colombia," Working papers 74, Red Investigadores de Economía.
    5. Islam, Asadul & Kwon, Sungoh & Masood, Eema & Prakash, Nishith & Sabarwal, Shwetlena & Saraswat, Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," IZA Discussion Papers 13188, Institute of Labor Economics (IZA).
    6. List, John A. & Livingston, Jeffrey A. & Neckermann, Susanne, 2018. "Do financial incentives crowd out intrinsic motivation to perform on standardized tests?," Economics of Education Review, Elsevier, vol. 66(C), pages 125-136.
    7. Islam, Asad & Kwon, Sungoh & Masood, Eema & Prakash, Nishith & Sabarwal, Shwetlena & Saraswat, Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," GLO Discussion Paper Series 526, Global Labor Organization (GLO).
    8. Calsamiglia, Caterina & Loviglio, Annalisa, 2019. "Grading on a curve: When having good peers is not good," Economics of Education Review, Elsevier, vol. 73(C).
    9. Alex Imas & Sally Sadoff & Anya Samek, 2017. "Do People Anticipate Loss Aversion?," Management Science, INFORMS, vol. 63(5), pages 1271-1284, May.
    10. Корешникова Ю. Н. & Захаров А. Б. & Дудырев Ф. Ф., 2018. "Различия В Общем Образовании В Колледжах И Старших Классах Школ: Характеристики Педагогов И Практики Преподавания. На Примере Математики," Вопросы образования // Educational Studies, НИУ ВШЭ, issue 2, pages 228-253.
    11. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    12. Czibor, Eszter & Onderstal, Sander & Sloof, Randolph & van Praag, C. Mirjam, 2020. "Does relative grading help male students? Evidence from a field experiment in the classroom," Economics of Education Review, Elsevier, vol. 75(C).
    13. Cathles, Alison & Ritzen, Jo, 2017. "Money Counts, but So Does Timing: Public Investment and Adult Competencies," IZA Discussion Papers 10565, Institute of Labor Economics (IZA).
    14. Crafts, Nicholas, 2017. "The Postwar British Productivity Failure," The Warwick Economics Research Paper Series (TWERPS) 1142, University of Warwick, Department of Economics.
    15. Bergbauer, Annika B., 2019. "How did EU membership of Eastern Europe affect student achievement?," EconStor Open Access Articles, ZBW - Leibniz Information Centre for Economics, pages 624-644.
    16. Silvia Griselda, 2020. "Different Questions, Different Gender Gap: Can the Format of Questions Explain the Gender Gap in Mathematics?," 2020 Papers pgr710, Job Market Papers.
    17. Balart, Pau & Oosterveen, Matthijs & Webbink, Dinand, 2018. "Test scores, noncognitive skills and economic growth," Economics of Education Review, Elsevier, vol. 63(C), pages 134-153.
    18. Contini, Dalit & Cugnata, Federica, 2018. "How do institutions affect learning inequalities? Revisiting difference-in-difference models with international assessments," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201817, University of Turin.
    19. Ludger Woessmann, 2018. "Central exit exams improve student outcomes," IZA World of Labor, Institute of Labor Economics (IZA), pages 419-419, January.
    20. Bastian Ravesteijn & Hans van Kippersluis & Mauricio Avendano & Pekka Martikainen & Hannu Vessari & Eddy van Doorslaer, 2017. "The Impact of Later Tracking on Mortality by Parental Income in Finland," Tinbergen Institute Discussion Papers 17-030/V, Tinbergen Institute.

    More about this item

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:24004. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.