IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v149y2018icp74-87.html
   My bibliography  Save this article

Experimental methods: Measuring effort in economics experiments

Author

Listed:
  • Charness, Gary
  • Gneezy, Uri
  • Henderson, Austin

Abstract

The study of effort provision in a controlled setting is a key research area in experimental economics. There are two major methodological paradigms in this literature: stated effort and real effort. In the stated-effort paradigm the researcher uses an “effort function” that maps choices to outcomes. In the real-effort paradigm, participants work on a task, and outcomes depend on their performance. The advantage of the stated-effort design is the control the researcher has over the cost of effort, which is particularly useful when testing theory. The advantage of the real-effort design is that it may be a better match to the field environment, particularly with respect to psychological aspects that affect behavior. An open question in the literature is the degree to which the results obtained by the two paradigms differ, and if they do, why. We present a review of methods used and discuss the results obtained from using these different approaches, and issues to consider when choosing and implementing a task.

Suggested Citation

  • Charness, Gary & Gneezy, Uri & Henderson, Austin, 2018. "Experimental methods: Measuring effort in economics experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 149(C), pages 74-87.
  • Handle: RePEc:eee:jeborg:v:149:y:2018:i:c:p:74-87
    DOI: 10.1016/j.jebo.2018.02.024
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268118300556
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jebo.2018.02.024?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jordi Brandts & David J. Cooper, 2007. "It's What You Say, Not What You Pay: An Experimental Study of Manager–Employee Relationships in Overcoming Coordination Failure," Journal of the European Economic Association, MIT Press, vol. 5(6), pages 1223-1268, December.
    2. Gary Charness, 2004. "Attribution and Reciprocity in an Experimental Labor Market," Journal of Labor Economics, University of Chicago Press, vol. 22(3), pages 665-688, July.
    3. Ernst Fehr & Georg Kirchsteiger & Arno Riedl, 1993. "Does Fairness Prevent Market Clearing? An Experimental Investigation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(2), pages 437-459.
    4. Charness, Gary & Dufwenberg, Martin, 2003. "Promises & Partnership," Research Papers in Economics 2003:3, Stockholm University, Department of Economics.
    5. Richard H. Thaler & Shlomo Benartzi, 2004. "Save More Tomorrow (TM): Using Behavioral Economics to Increase Employee Saving," Journal of Political Economy, University of Chicago Press, vol. 112(S1), pages 164-187, February.
    6. Benny Moldovanu & Aner Sela, 2008. "The Optimal Allocation of Prizes in Contests," Springer Books, in: Roger D. Congleton & Arye L. Hillman & Kai A. Konrad (ed.), 40 Years of Research on Rent Seeking 1, pages 615-631, Springer.
    7. Ernst Fehr & Simon Gachter & Georg Kirchsteiger, 1997. "Reciprocity as a Contract Enforcement Device: Experimental Evidence," Econometrica, Econometric Society, vol. 65(4), pages 833-860, July.
    8. Uri Gneezy & Aldo Rustichini, 2000. "Pay Enough or Don't Pay at All," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(3), pages 791-810.
    9. Alwine Mohnen & Kathrin Pokorny & Dirk Sliwka, 2008. "Transparency, Inequity Aversion, and the Dynamics of Peer Pressure in Teams: Theory and Evidence," Journal of Labor Economics, University of Chicago Press, vol. 26(4), pages 693-720, October.
    10. Johannes Abeler & Armin Falk & Lorenz Goette & David Huffman, 2011. "Reference Points and Effort Provision," American Economic Review, American Economic Association, vol. 101(2), pages 470-492, April.
    11. Lévy-Garboua, Louis & Masclet, David & Montmarquette, Claude, 2009. "A behavioral Laffer curve: Emergence of a social norm of fairness in a real effort experiment," Journal of Economic Psychology, Elsevier, vol. 30(2), pages 147-161, April.
    12. Heike Hennig-Schmidt & Bettina Rockenbach & Abdolkarim Sadrieh, 2010. "In Search Of Workers' Real Effort Reciprocity-A Field and a Laboratory Experiment," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 817-837, June.
    13. Giovanna Devetag & Andreas Ortmann, 2007. "When and why? A critical survey on coordination failure in the laboratory," Experimental Economics, Springer;Economic Science Association, vol. 10(3), pages 331-344, September.
    14. Van Huyck, John B & Battalio, Raymond C & Beil, Richard O, 1990. "Tacit Coordination Games, Strategic Uncertainty, and Coordination Failure," American Economic Review, American Economic Association, vol. 80(1), pages 234-248, March.
    15. Martin Brown & Armin Falk & Ernst Fehr, 2004. "Relational Contracts and the Nature of Market Interactions," Econometrica, Econometric Society, vol. 72(3), pages 747-780, May.
    16. Katharina Laske & Marina Schroeder, 2016. "Quantity, Quality, and Originality: The Effects of Incentives on Creativity," Cologne Graduate School Working Paper Series 07-01, Cologne Graduate School in Management, Economics and Social Sciences.
    17. Sebastian Kube & Michel André Maréchal & Clemens Puppe, 2013. "Do Wage Cuts Damage Work Morale? Evidence From A Natural Field Experiment," Journal of the European Economic Association, European Economic Association, vol. 11(4), pages 853-870, August.
    18. Offerman, Theo & van der Veen, Ailko, 2015. "How to subsidize contributions to public goods: Does the frog jump out of the boiling water?," European Economic Review, Elsevier, vol. 74(C), pages 96-108.
    19. Alexander L. Brown & Jonathan Meer & J. Forrest Williams, 2019. "Why Do People Volunteer? An Experimental Analysis of Preferences for Time Donations," Management Science, INFORMS, vol. 65(4), pages 1455-1468, April.
    20. Jordi Brandts & Matthew Ellman & Gary Charness, 2016. "Let’s Talk: How Communication Affects Contract Design," Journal of the European Economic Association, European Economic Association, vol. 14(4), pages 943-974.
    21. Gary Charness & Marie-Claire Villeval, 2009. "Cooperation and Competition in Intergenerational Experiments in the Field and the Laboratory," American Economic Review, American Economic Association, vol. 99(3), pages 956-978, June.
    22. Nicolas Houy & Jean-Philippe Nicolaï & Marie Claire Villeval, 2016. "Doing Your Best when Stakes are High? Theory and Experimental Evidence," Working Papers halshs-01277982, HAL.
    23. Michael Kosfeld & Susanne Neckermann, 2011. "Getting More Work for Nothing? Symbolic Awards and Worker Performance," American Economic Journal: Microeconomics, American Economic Association, vol. 3(3), pages 86-99, August.
    24. Bisin, Alberto & Hyndman, Kyle, 2020. "Present-bias, procrastination and deadlines in a field experiment," Games and Economic Behavior, Elsevier, vol. 119(C), pages 339-357.
    25. Emanuela Lezzi & Piers Fleming & Daniel John Zizzo, 2015. "Does it matter which effort task you use? A comparison of four effort tasks when agents compete for a prize," Working Paper series, University of East Anglia, Centre for Behavioural and Experimental Social Science (CBESS) 15-05, School of Economics, University of East Anglia, Norwich, UK..
    26. Victoria Prowse & David Gill, 2009. "A Novel Computerized Real Effort Task Based on Sliders," Economics Series Working Papers 435, University of Oxford, Department of Economics.
    27. van Dijk, Frans & Sonnemans, Joep & van Winden, Frans, 2001. "Incentive systems in a real effort experiment," European Economic Review, Elsevier, vol. 45(2), pages 187-214, February.
    28. Ned Augenblick & Muriel Niederle & Charles Sprenger, 2015. "Editor's Choice Working over Time: Dynamic Inconsistency in Real Effort Tasks," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1067-1115.
    29. Ananish Chaudhuri & Andrew Schotter & Barry Sopher, 2009. "Talking Ourselves to Efficiency: Coordination in Inter‐Generational Minimum Effort Games with Private, Almost Common and Common Knowledge of Advice," Economic Journal, Royal Economic Society, vol. 119(534), pages 91-122, January.
    30. Femida Handy & Eliakim Katz, 2008. "Donating behavior: if time is money, which to give? A preliminary analysis," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 35(4), pages 323-332, September.
    31. Steven J. Kachelmeier & Bernhard E. Reichert & Michael G. Williamson, 2008. "Measuring and Motivating Quantity, Creativity, or Both," Journal of Accounting Research, Wiley Blackwell, vol. 46(2), pages 341-373, May.
    32. Claude Montmarquette & Jean-Louis Rullière & Marie-Claire Villeval & Romain Zeiliger, 2004. "Redesigning Teams and Incentives in a Merger: An Experiment with Managers and Students," Management Science, INFORMS, vol. 50(10), pages 1379-1389, October.
    33. Charness, Gary, 2000. "Responsibility and effort in an experimental labor market," Journal of Economic Behavior & Organization, Elsevier, vol. 42(3), pages 375-384, July.
    34. Simon Gächter & Lingbo Huang & Martin Sefton, 2016. "Combining “real effort” with induced effort costs: the ball-catching task," Experimental Economics, Springer;Economic Science Association, vol. 19(4), pages 687-712, December.
    35. Victoria Prowse & David Gill, 2009. "A Novel Computerized Real Effort Task Based on Sliders," Economics Series Working Papers 435, University of Oxford, Department of Economics.
    36. Johnson, Noel D. & Mislin, Alexandra A., 2011. "Trust games: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 32(5), pages 865-889.
    37. Gary Charness & Martin Dufwenberg, 2006. "Promises and Partnership," Econometrica, Econometric Society, vol. 74(6), pages 1579-1601, November.
    38. Felipe Kast & Stephan Meier & Dina Pomeranz, 2012. "Under-Savers Anonymous: Evidence on Self-Help Groups and Peer Pressure as a Savings Commitment Device," NBER Working Papers 18417, National Bureau of Economic Research, Inc.
    39. Muriel Niederle & Lise Vesterlund, 2007. "Do Women Shy Away From Competition? Do Men Compete Too Much?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1067-1101.
    40. Charness, Gary & Kuhn, Peter, 2011. "Lab Labor: What Can Labor Economists Learn from the Lab?," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 3, pages 229-330, Elsevier.
    41. Brice Corgnet & Roberto Hernán-González & Stephen Rassenti, 2011. "Real Effort, Real Leisure and Real-time Supervision: Incentives and Peer Pressure in Virtual Organizations," Working Papers 11-05, Chapman University, Economic Science Institute.
    42. Gary Charness & Daniela Grieco, 2019. "Creativity and Incentives," Journal of the European Economic Association, European Economic Association, vol. 17(2), pages 454-496.
    43. Andreoni, James, 1990. "Impure Altruism and Donations to Public Goods: A Theory of Warm-Glow Giving?," Economic Journal, Royal Economic Society, vol. 100(401), pages 464-477, June.
    44. Dickinson, David L, 1999. "An Experimental Examination of Labor Supply and Work Intensities," Journal of Labor Economics, University of Chicago Press, vol. 17(4), pages 638-670, October.
    45. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    46. Dutcher, E. Glenn & Salmon, Timothy C. & Saral, Krista J., 2015. "Is "Real" Effort More Real?," MPRA Paper 68394, University Library of Munich, Germany.
    47. Bruggen, Alexander & Strobel, Martin, 2007. "Real effort versus chosen effort in experiments," Economics Letters, Elsevier, vol. 96(2), pages 232-236, August.
    48. James Konow, 2000. "Fair Shares: Accountability and Cognitive Dissonance in Allocation Decisions," American Economic Review, American Economic Association, vol. 90(4), pages 1072-1091, September.
    49. Smith, Vernon L, 1976. "Experimental Economics: Induced Value Theory," American Economic Review, American Economic Association, vol. 66(2), pages 274-279, May.
    50. Gary Charness & Ramón Cobo-Reyes & Juan A. Lacomba & Francisco Lagos & Jose Maria Perez, 2016. "Social comparisons in wage delegation: experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 19(2), pages 433-459, June.
    51. Rutstrom, E. Elisabet & Williams, Melonie B., 2000. "Entitlements and fairness:: an experimental study of distributive preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 43(1), pages 75-89, September.
    52. Uri Gneezy & Muriel Niederle & Aldo Rustichini, 2003. "Performance in Competitive Environments: Gender Differences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 1049-1074.
    53. Stefania Bortolotti & Giovanna Devetag & Andreas Ortmann, 2009. "Exploring the effects of real effort in a weak-link experiment," CEEL Working Papers 0901, Cognitive and Experimental Economics Laboratory, Department of Economics, University of Trento, Italia.
    54. Gary Charness & Uri Gneezy, 2009. "Incentives to Exercise," Econometrica, Econometric Society, vol. 77(3), pages 909-931, May.
    55. Uri Gneezy & John A List, 2006. "Putting Behavioral Economics to Work: Testing for Gift Exchange in Labor Markets Using Field Experiments," Econometrica, Econometric Society, vol. 74(5), pages 1365-1384, September.
    56. Thomas Buser & Noemi Peter, 2012. "Multitasking," Experimental Economics, Springer;Economic Science Association, vol. 15(4), pages 641-655, December.
    57. Ananish Chaudhuri, 2011. "Sustaining cooperation in laboratory public goods experiments: a selective survey of the literature," Experimental Economics, Springer;Economic Science Association, vol. 14(1), pages 47-83, March.
    58. Charness, Gary & Cobo-Reyes, Ramón & Sánchez, Ángela, 2016. "The effect of charitable giving on workers’ performance: Experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 131(PA), pages 61-74.
    59. Sanjiv Erat & Uri Gneezy, 2016. "Incentives for creativity," Experimental Economics, Springer;Economic Science Association, vol. 19(2), pages 269-280, June.
    60. Swenson, Charles W., 1988. "Taxpayer behavior in response to taxation: An experimental analysis," Journal of Accounting and Public Policy, Elsevier, vol. 7(1), pages 1-28.
    61. Uri Gneezy & Aldo Rustichini, 2004. "Gender and Competition at a Young Age," American Economic Review, American Economic Association, vol. 94(2), pages 377-381, May.
    62. Bull, Clive & Schotter, Andrew & Weigelt, Keith, 1987. "Tournaments and Piece Rates: An Experimental Study," Journal of Political Economy, University of Chicago Press, vol. 95(1), pages 1-33, February.
    63. Dan Ariely & Uri Gneezy & George Loewenstein & Nina Mazar, 2009. "Large Stakes and Big Mistakes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 451-469.
    64. Wieland Müller & Andrew Schotter, 2010. "Workaholics and Dropouts in Organizations," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 717-743, June.
    65. Nisvan Erkal & Lata Gangadharan & Nikos Nikiforakis, 2011. "Relative Earnings and Giving in a Real-Effort Experiment," American Economic Review, American Economic Association, vol. 101(7), pages 3330-3348, December.
    66. Imas, Alex, 2014. "Working for the “warm glow”: On the benefits and limits of prosocial incentives," Journal of Public Economics, Elsevier, vol. 114(C), pages 14-18.
    67. Gross, Till & Guo, Christopher & Charness, Gary, 2015. "Merit pay and wage compression with productivity differences and uncertainty," Journal of Economic Behavior & Organization, Elsevier, vol. 117(C), pages 233-247.
    68. Fahr, Rene & Irlenbusch, Bernd, 2000. "Fairness as a constraint on trust in reciprocity: earned property rights in a reciprocal exchange experiment," Economics Letters, Elsevier, vol. 66(3), pages 275-282, March.
    69. Offerman, Theo, 2002. "Hurting hurts more than helping helps," European Economic Review, Elsevier, vol. 46(8), pages 1423-1437, September.
    70. Karan Girotra & Christian Terwiesch & Karl T. Ulrich, 2010. "Idea Generation and the Quality of the Best Idea," Management Science, INFORMS, vol. 56(4), pages 591-605, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charness, Gary & Kuhn, Peter, 2011. "Lab Labor: What Can Labor Economists Learn from the Lab?," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 3, pages 229-330, Elsevier.
    2. Emanuela Lezzi & Piers Fleming & Daniel John Zizzo, 2015. "Does it matter which effort task you use? A comparison of four effort tasks when agents compete for a prize," Working Paper series, University of East Anglia, Centre for Behavioural and Experimental Social Science (CBESS) 15-05, School of Economics, University of East Anglia, Norwich, UK..
    3. Dutcher, E. Glenn & Salmon, Timothy C. & Saral, Krista J., 2015. "Is "Real" Effort More Real?," MPRA Paper 68394, University Library of Munich, Germany.
    4. Erkal, Nisvan & Gangadharan, Lata & Koh, Boon Han, 2018. "Monetary and non-monetary incentives in real-effort tournaments," European Economic Review, Elsevier, vol. 101(C), pages 528-545.
    5. Gill, David & Prowse, Victoria, 2019. "Measuring costly effort using the slider task," Journal of Behavioral and Experimental Finance, Elsevier, vol. 21(C), pages 1-9.
    6. Simon Gächter & Lingbo Huang & Martin Sefton, 2016. "Combining “real effort” with induced effort costs: the ball-catching task," Experimental Economics, Springer;Economic Science Association, vol. 19(4), pages 687-712, December.
    7. Brice Corgnet & Roberto Hernán-González & Eric Schniter, 2015. "Why real leisure really matters: incentive effects on real effort in the laboratory," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 284-301, June.
    8. Bejarano, Hernán & Corgnet, Brice & Gómez-Miñambres, Joaquín, 2021. "Economic stability promotes gift-exchange in the workplace," Journal of Economic Behavior & Organization, Elsevier, vol. 187(C), pages 374-398.
    9. Felipe Augusto de Araujo & Erin Carbone & Lynn Conell-Price & Marli W. Dunietz & Ania Jaroszewicz & Rachel Landsman & Diego Lamé & Lise Vesterlund & Stephanie Wang & Alistair J. Wilson, 2015. "The Effect of Incentives on Real Effort: Evidence from the Slider Task," CESifo Working Paper Series 5372, CESifo.
    10. Dohmen, Thomas, 2014. "Behavioral labor economics: Advances and future directions," Labour Economics, Elsevier, vol. 30(C), pages 71-85.
    11. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    12. Faralla, Valeria & Borà, Guido & Innocenti, Alessandro & Novarese, Marco, 2020. "Promises in group decision making," Research in Economics, Elsevier, vol. 74(1), pages 1-11.
    13. Gary Charness & Ramón Cobo-Reyes & Juan A. Lacomba & Francisco Lagos & Jose Maria Perez, 2016. "Social comparisons in wage delegation: experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 19(2), pages 433-459, June.
    14. Victoria Prowse & David Gill, 2009. "A Novel Computerized Real Effort Task Based on Sliders," Economics Series Working Papers 435, University of Oxford, Department of Economics.
    15. Gagnon, Nickolas & Noussair, C., 2016. "Does Reciprocity Persist Over Time?," Research Memorandum 033, Maastricht University, Graduate School of Business and Economics (GSBE).
    16. Boosey, Luke & Goerg, Sebastian, 2020. "The timing of discretionary bonuses – effort, signals, and reciprocity," Games and Economic Behavior, Elsevier, vol. 124(C), pages 254-280.
    17. Jared Rubin & Roman Sheremeta, 2016. "Principal–Agent Settings with Random Shocks," Management Science, INFORMS, vol. 62(4), pages 985-999, April.
    18. Sabrina Jeworrek & Vanessa Mertins, 2019. "Wage delegation in the field," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 28(4), pages 656-669, November.
    19. Cardella, Eric & Depew, Briggs, 2018. "Output restriction and the ratchet effect: Evidence from a real-effort work task," Games and Economic Behavior, Elsevier, vol. 107(C), pages 182-202.
    20. Bradler, Christiane & Neckermann, Susanne & Warnke, Arne Jonas, 2016. "Incentivizing creativity: A large-scale experiment with tournaments and gifts," ZEW Discussion Papers 16-040, ZEW - Leibniz Centre for European Economic Research.

    More about this item

    Keywords

    Stated effort; Real effort; Timing of decisions; Goal-orientation; Experimental methodology;
    All these keywords.

    JEL classification:

    • B49 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Other
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:149:y:2018:i:c:p:74-87. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.