IDEAS home Printed from https://ideas.repec.org/a/bla/buecrs/v77y2025i2p198-218.html

Productivity and Task Heterogeneity in Online Labor Markets: A Bonus Payment Experiment

Author

Listed:
  • Evaggelos Mourelatos
  • Nicholas Giannakopoulos
  • Manolis Tzagarakis

Abstract

We use an online experiment to investigate how monetary rewards and within‐task heterogeneity interact with workers’ productivity. By uniquely conceptualizing objective difficulty and complexity, we first investigate whether within‐task heterogeneity results in responses that are elicited solely by task characteristics related to the computational efficiency of the task doer. The subtasks’ difficulty and complexity have a robust reverse effect on every productivity outcome of the participant workers. Second, we find that monetary rewards mainly act as moderators of within‐task heterogeneity. Our investigation identified monetary rewards as a key moderator of within‐task heterogeneity. Notably, higher rewards effectively counteract the detrimental impact of task complexity and difficulty, yet we observe diminishing returns with extreme reward values. The study's central contribution lies in its uncovering of a complementary relationship. An optimally designed task coupled with a judicious reward policy not only eases workers’ perceived complexity and difficulty but also alleviates uncertainties that might otherwise hinder the achievement of high‐quality outcomes. This nuanced understanding provides valuable insights for shaping efficient task structures and reward strategies in the realm of online labor markets. An analysis of almost 20,396 crowdsourcing responses supports the theoretical model and hypotheses.

Suggested Citation

  • Evaggelos Mourelatos & Nicholas Giannakopoulos & Manolis Tzagarakis, 2025. "Productivity and Task Heterogeneity in Online Labor Markets: A Bonus Payment Experiment," Bulletin of Economic Research, Wiley Blackwell, vol. 77(2), pages 198-218, April.
  • Handle: RePEc:bla:buecrs:v:77:y:2025:i:2:p:198-218
    DOI: 10.1111/boer.12476
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/boer.12476
    Download Restriction: no

    File URL: https://libkey.io/10.1111/boer.12476?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    2. Shilpi Jain & Swanand J. Deodhar, 2022. "Social mechanisms in crowdsourcing contests: a literature review," Behaviour and Information Technology, Taylor & Francis Journals, vol. 41(5), pages 1080-1114, April.
    3. Thomas Dohmen & Armin Falk, 2011. "Performance Pay and Multidimensional Sorting: Productivity, Preferences, and Gender," American Economic Review, American Economic Association, vol. 101(2), pages 556-590, April.
    4. Maria Cubel & Ana Nuevo‐Chiquero & Santiago Sanchez‐Pages & Marian Vidal‐Fernandez, 2016. "Do Personality Traits Affect Productivity? Evidence from the Laboratory," Economic Journal, Royal Economic Society, vol. 0(592), pages 654-681, May.
    5. Bruce Shearer, 2004. "Piece Rates, Fixed Wages and Incentives: Evidence from a Field Experiment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 71(2), pages 513-534.
    6. Muriel Niederle & Lise Vesterlund, 2007. "Do Women Shy Away From Competition? Do Men Compete Too Much?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1067-1101.
    7. Butschek, Sebastian & González Amor, Roberto & Kampkötter, Patrick & Sliwka, Dirk, 2022. "Motivating gig workers – evidence from a field experiment," Labour Economics, Elsevier, vol. 75(C).
    8. Andrei Bremzen & Elena Khokhlova & Anton Suvorov & Jeroen van de Ven, 2015. "Bad News: An Experimental Study on the Informational Effects Of Rewards," The Review of Economics and Statistics, MIT Press, vol. 97(1), pages 55-70, March.
    9. Kathrin Manthei & Dirk Sliwka & Timo Vogelsang, 2021. "Performance Pay and Prior Learning—Evidence from a Retail Chain," Management Science, INFORMS, vol. 67(11), pages 6998-7022, November.
    10. Cantarella, Michele & Strozzi, Chiara, 2022. "Piecework and Job Search in the Platform Economy," IZA Discussion Papers 15775, IZA Network @ LISER.
    11. Guido Friebel & Matthias Heinz & Miriam Krueger & Nikolay Zubanov, 2017. "Team Incentives and Performance: Evidence from a Retail Chain," American Economic Review, American Economic Association, vol. 107(8), pages 2168-2203, August.
    12. Mourelatos, Evangelos, 2023. "Does Mood affect Sexual and Gender Discrimination in Hiring Choices? Evidence from Online Experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    13. Alberto Dalmazzo, 2002. "Technological Complexity, Wage Differentials and Unemployment," Scandinavian Journal of Economics, Wiley Blackwell, vol. 104(4), pages 515-530, December.
    14. repec:eee:respol:v:48:y:2019:i:8:p:- is not listed on IDEAS
    15. Joni Salminen & Ahmed Mohamed Sayed Kamel & Soon-Gyo Jung & Mekhail Mustak & Bernard J. Jansen, 2023. "Fair compensation of crowdsourcing work: the problem of flat rates," Behaviour and Information Technology, Taylor & Francis Journals, vol. 42(16), pages 2871-2892, December.
    16. Wei Wu & Xiang Gong & Qianwen Yang, 2023. "Role of motivations, self-regulations, and perceived competitive intensity in solvers’ continuance intention in crowdsourcing contests," Behaviour and Information Technology, Taylor & Francis Journals, vol. 42(13), pages 2152-2175, October.
    17. Roger Newson, 2004. "Generalized power calculations for generalized linear models and more," Stata Journal, StataCorp LLC, vol. 4(4), pages 379-401, December.
    18. Aleksandr Alekseev, 2022. "Give me a challenge or give me a raise," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 170-202, February.
    19. Berinsky, Adam J. & Huber, Gregory A. & Lenz, Gabriel S., 2012. "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk," Political Analysis, Cambridge University Press, vol. 20(3), pages 351-368, July.
    20. repec:bla:scandj:v:104:y:2002:i:4:p:515-30 is not listed on IDEAS
    21. Michael Gibbs & Susanne Neckermann & Christoph Siemroth, 2017. "A Field Experiment in Motivating Employee Ideas," The Review of Economics and Statistics, MIT Press, vol. 99(4), pages 577-590, July.
    22. List, John A. & Momeni, Fatemeh, 2020. "Leveraging upfront payments to curb employee misbehavior: Evidence from a natural field experiment," European Economic Review, Elsevier, vol. 130(C).
    23. Shintaro Yamaguchi, 2012. "Tasks and Heterogeneous Human Capital," Journal of Labor Economics, University of Chicago Press, vol. 30(1), pages 1-53.
    24. Heckman, James J & Sedlacek, Guilherme, 1985. "Heterogeneity, Aggregation, and Market Wage Functions: An Empirical Model of Self-selection in the Labor Market," Journal of Political Economy, University of Chicago Press, vol. 93(6), pages 1077-1125, December.
    25. Matthew J C Crump & John V McDonnell & Todd M Gureckis, 2013. "Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-18, March.
    26. Acar, Oguz A., 2019. "Motivations and solution appropriateness in crowdsourcing challenges for innovation," Research Policy, Elsevier, vol. 48(8), pages 1-1.
    27. Evangelos Mourelatos & George Krimpas & Konstantinos Giotopoulos, 2024. "Sexual Identity and Gender Gap in Political Leadership Ambition: An Experiment," Review of Behavioral Economics, now publishers, vol. 11(1), pages 73-121, February.
    28. Harry J. Paarsch & Bruce S. Shearer, 1999. "The Response of Worker Effort to Piece Rates: Evidence from the British Columbia Tree-Planting Industry," Journal of Human Resources, University of Wisconsin Press, vol. 34(4), pages 643-667.
    29. David Johnson & John Barry Ryan, 2020. "Amazon Mechanical Turk workers can provide consistent and economically meaningful data," Southern Economic Journal, John Wiley & Sons, vol. 87(1), pages 369-385, July.
    30. Daniel L. Chen & John J. Horton, 2016. "Research Note—Are Online Labor Markets Spot Markets for Tasks? A Field Experiment on the Behavioral Response to Wage Cuts," Information Systems Research, INFORMS, vol. 27(2), pages 403-423, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mourelatos, Evangelos & Zervas, Panagiotis & Lagios, Dimitris & Tzimas, Giannis, 2024. "Can AI Bridge the Gender Gap in Competitiveness?," GLO Discussion Paper Series 1404, Global Labor Organization (GLO).
    2. Saskia Opitz & Dirk Sliwka & Timo Vogelsang & Tom Zimmermann, 2025. "The Algorithmic Assignment of Incentive Schemes," Management Science, INFORMS, vol. 71(2), pages 1546-1563, February.
    3. Michael Gibbs & Friederike Mengel & Christoph Siemroth, 2023. "Work from Home and Productivity: Evidence from Personnel and Analytics Data on Information Technology Professionals," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 1(1), pages 7-41.
    4. Saskia Opitz & Dirk Sliwka & Timo Vogelsang & Tom Zimmermann, 2022. "The Targeted Assignment of Incentive Schemes," ECONtribute Discussion Papers Series 187, University of Bonn and University of Cologne, Germany.
    5. Johannes G. Jaspersen & Marc A. Ragin & Justin R. Sydnor, 2022. "Insurance demand experiments: Comparing crowdworking to the lab," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 89(4), pages 1077-1107, December.
    6. Evangelos Mourelatos & Nicholas Giannakopoulos & Manolis Tzagarakis, 2022. "Personality traits and performance in online labour markets," Behaviour and Information Technology, Taylor & Francis Journals, vol. 41(3), pages 468-484, February.
    7. Mourelatos, Evangelos & Krimpas, George & Giotopoulos, Konstantinos, 2022. "Sexual identity and Gender Gap in Leadership. A political intention experiment," GLO Discussion Paper Series 1187, Global Labor Organization (GLO).
    8. Sebastian Fest & Ola Kvaløy & Petra Nieken & Anja Schöttner, 2019. "Motivation and incentives in an online labor market," CESifo Working Paper Series 7526, CESifo.
    9. Ku, Hyejin, 2019. "The effect of wage subsidies on piece rate workers: Evidence from the Penny Per Pound program in Florida," Journal of Development Economics, Elsevier, vol. 139(C), pages 122-134.
    10. Englmaier, Florian & Grimm, Stefan & Schindler, David & Schudy, Simeon, 2018. "The Effect of Incentives in Non-Routine Analytical Team Tasks – Evidence from a Field Experiment," VfS Annual Conference 2017 (Vienna): Alternative Structures for Money and Banking 168286, Verein für Socialpolitik / German Economic Association.
    11. Oliver Fabel & Sandra Mauser & Yingchao Zhang, 2024. "Performance contests and merit pay with empathic employees," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 45(1), pages 353-372, January.
    12. Clemens, Marco & Sauermann, Jan, 2025. "Making the right call: The heterogeneous effects of individual performance pay on productivity," Labour Economics, Elsevier, vol. 94(C).
    13. Balbuzanov, Ivan & Gars, Jared & Stalinski, Mateusz & Tjernström, Emilia, 2025. "Incentivizing Engagement: Experimental Evidence on Journalist Performance Pay," CAGE Online Working Paper Series 763, Competitive Advantage in the Global Economy (CAGE).
    14. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," IZA Discussion Papers 15478, IZA Network @ LISER.
    15. Jakob Alfitian & Dirk Sliwka & Timo Vogelsang, 2024. "When Bonuses Backfire: Evidence from the Workplace," Management Science, INFORMS, vol. 70(9), pages 6395-6414, September.
    16. Haas, Nicholas & Hassan, Mazen & Mansour, Sarah & Morton, Rebecca B., 2021. "Polarizing information and support for reform," Journal of Economic Behavior & Organization, Elsevier, vol. 185(C), pages 883-901.
    17. Joseph A. Johnson & Jochen Theis & Adam Vitalis & Donald Young, 2020. "The Influence of Firms' Emissions Management Strategy Disclosures on Investors' Valuation Judgments†," Contemporary Accounting Research, John Wiley & Sons, vol. 37(2), pages 642-664, June.
    18. Gandullia, Luca & Lezzi, Emanuela & Parciasepe, Paolo, 2020. "Replication with MTurk of the experimental design by Gangadharan, Grossman, Jones & Leister (2018): Charitable giving across donor types," Journal of Economic Psychology, Elsevier, vol. 78(C).
    19. Adrian Bruhin & Fidel Petros & Luís Santos-Pinto, 2024. "The role of self-confidence in teamwork: experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 27(3), pages 687-712, July.
    20. Jochen Theis & Marvin Nipper & Marco Meier, 2024. "The influence of corporate philanthropic donations on private investors' valuation judgments: Experimental evidence," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 31(1), pages 540-554, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:buecrs:v:77:y:2025:i:2:p:198-218. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0307-3378 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.