IDEAS home Printed from https://ideas.repec.org/a/inm/orstsc/v10y2025i4p421-444.html

Motives, Gender, and Experience: Performance Effects in Crowdsourcing Contests

Author

Listed:
  • Jonas Heite

    (Max Planck Institute for Innovation and Competition, 80333 Munich, Germany)

  • Karin Hoisl

    (Max Planck Institute for Innovation and Competition, 80333 Munich, Germany; and University of Mannheim, Schloss, 68161 Mannheim, Germany; and Department of Strategy and Innovation, Copenhagen Business School, 2000 Frederiksberg, Denmark)

  • Rainer Widmann

    (Max Planck Institute for Innovation and Competition, 80333 Munich, Germany)

Abstract

Our study examines how individual characteristics—economic versus achievement-based motives, gender, and experience—moderate the “performance revision effect” in tournament-based crowdsourcing competitions. This effect refers to a phenomenon in which contestants reduce their effort when competing against significantly higher-ability opponents. Using data from Topcoder, a leading crowdsourcing platform, we conducted a quasiexperimental study with 1,677 coders in 38 single-round matches. Our regression discontinuity design exploits Topcoder’s skill-based divisions to assess contestants’ responses to differing opponent abilities. The results confirm the performance revision effect, revealing an average performance decline of 20% when contestants face higher-ability opponents. Moreover, female and more experienced participants show a stronger response to the performance revision effect than their male and less-experienced peers. Our findings contribute to the crowdsourcing literature by highlighting the boundary conditions of the performance revision effect and by quantifying the performance implications of contest designs for different contestants, allowing platform operators to make data-driven cost-benefit decisions about contest design to mitigate performance losses.

Suggested Citation

  • Jonas Heite & Karin Hoisl & Rainer Widmann, 2025. "Motives, Gender, and Experience: Performance Effects in Crowdsourcing Contests," Strategy Science, INFORMS, vol. 10(4), pages 421-444, December.
    • Handle: RePEc:inm:orstsc:v:10:y:2025:i:4:p:421-444
    DOI: 10.1287/stsc.2023.0068
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/stsc.2023.0068
    Download Restriction: no
    File URL: https://libkey.io/10.1287/stsc.2023.0068?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Catherine C. Eckel & Philip J. Grossman, 2002. "Sex Differences and Statistical Stereotyping in Attitudes Toward Financial Risk," Monash Economics Working Papers archive-03, Monash University, Department of Economics.
    2. Jennifer Brown, 2011. "Quitters Never Win: The (Adverse) Incentive Effects of Competing with Superstars," Journal of Political Economy, University of Chicago Press, vol. 119(5), pages 982-1013.
    3. Muriel Niederle & Lise Vesterlund, 2011. "Gender and Competition," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 601-630, September.
    4. Konrad, Kai A., 2009. "Strategy and Dynamics in Contests," OUP Catalogue, Oxford University Press, number 9780199549603.
    5. Kevin J. Boudreau & Karim R. Lakhani & Michael Menietti, 2016. "Performance responses to competition across skill levels in rank-order tournaments: field evidence and implications for tournament design," RAND Journal of Economics, RAND Corporation, vol. 47(1), pages 140-165, February.
    6. Lorenzo Spadoni & Jan Potters, 2018. "The Effect of Competition on Risk Taking in Contests," Games, MDPI, vol. 9(3), pages 1-18, September.
    7. Ryan Oprea & Daniel Friedman & Steven T. Anderson, 2009. "Learning to Wait: A Laboratory Investigation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1103-1124.
    8. Lallemand, Thierry & Plasman, Robert & Rycx, François, 2005. "Women and Competition in Elimination Tournaments: Evidence from Professional Tennis Data," IZA Discussion Papers 1843, IZA Network @ LISER.
    9. Josh Lerner & Jean Tirole, 2002. "Some Simple Economics of Open Source," Journal of Industrial Economics, Wiley Blackwell, vol. 50(2), pages 197-234, June.
    10. Andrew Schotter & Keith Weigelt, 1992. "Asymmetric Tournaments, Equal Opportunity Laws, and Affirmative Action: Some Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(2), pages 511-539.
    11. Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2020. "Simple Local Polynomial Density Estimators," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(531), pages 1449-1455, July.
    12. Konrad, Kai A. & Lommerud, Kjell Erik, 1993. "Relative standing comparisons, risk taking, and safety regulations," Journal of Public Economics, Elsevier, vol. 51(3), pages 345-358, July.
    13. Lei Xu & Tingting Nian & Luís Cabral, 2020. "What Makes Geeks Tick? A Study of Stack Overflow Careers," Management Science, INFORMS, vol. 66(2), pages 587-604, February.
    14. Ming Hu & Lu Wang, 2021. "Joint vs. Separate Crowdsourcing Contests," Management Science, INFORMS, vol. 67(5), pages 2711-2728, May.
    15. Laurie S. Kirsch, 1997. "Portfolios of Control Modes and IS Project Management," Information Systems Research, INFORMS, vol. 8(3), pages 215-239, September.
    16. Sunil Mithas & M. S. Krishnan, 2008. "Human Capital and Institutional Effects in the Compensation of Information Technology Professionals in the United States," Management Science, INFORMS, vol. 54(3), pages 415-428, March.
    17. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell & Roc ́ıo Titiunik, 2017. "rdrobust: Software for regression-discontinuity designs," Stata Journal, StataCorp LLC, vol. 17(2), pages 372-404, June.
    18. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    19. Thierry Lallemand & Robert Plasman & François Rycx, 2008. "Women and Competition in Elimination Tournaments," Journal of Sports Economics, , vol. 9(1), pages 3-19, February.
    20. Jehiel, Philippe, 2005. "Analogy-based expectation equilibrium," Journal of Economic Theory, Elsevier, vol. 123(2), pages 81-104, August.
    21. Fonseca, Miguel A., 2009. "An experimental investigation of asymmetric contests," International Journal of Industrial Organization, Elsevier, vol. 27(5), pages 582-591, September.
    22. Iacus, Stefano & King, Gary & Porro, Giuseppe, 2009. "cem: Software for Coarsened Exact Matching," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 30(i09).
    23. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    24. Matthew Blackwell & Stefano Iacus & Gary King & Giuseppe Porro, 2009. "cem: Coarsened exact matching in Stata," Stata Journal, StataCorp LLC, vol. 9(4), pages 524-546, December.
    25. Douglas Davis & Robert Reilly, 1998. "Do too many cooks always spoil the stew? An experimental analysis of rent-seeking and the role of a strategic buyer," Public Choice, Springer, vol. 95(1), pages 89-115, April.
    26. Sonali K. Shah, 2006. "Motivation, Governance, and the Viability of Hybrid Forms in Open Source Software Development," Management Science, INFORMS, vol. 52(7), pages 1000-1014, July.
    27. Sebastian Calonico & Matias D. Cattaneo & Rocío Titiunik, 2015. "Optimal Data-Driven Regression Discontinuity Plots," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1753-1769, December.
    28. Anderson, Lisa R & Stafford, Sarah L, 2003. "An Experimental Analysis of Rent Seeking under Varying Competitive Conditions," Public Choice, Springer, vol. 115(1-2), pages 199-216, April.
    29. Thomas Buser, 2016. "The Impact of Losing in a Competition on the Willingness to Seek Further Challenges," Management Science, INFORMS, vol. 62(12), pages 3439-3449, December.
    30. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    31. Kagel, John H., 1995. "Cross-game learning: Experimental evidence from first-price and English common value auctions," Economics Letters, Elsevier, vol. 49(2), pages 163-170, August.
    32. Muriel Niederle & Lise Vesterlund, 2007. "Do Women Shy Away From Competition? Do Men Compete Too Much?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1067-1101.
    33. Kagel, John H & Harstad, Ronald M & Levin, Dan, 1987. "Information Impact and Allocation Rules in Auctions with Affiliated Private Values: A Laboratory Study," Econometrica, Econometric Society, vol. 55(6), pages 1275-1304, November.
    34. Patel, Chirag & Ahmad Husairi, Mariyani & Haon, Christophe & Oberoi, Poonam, 2023. "Monetary rewards and self-selection in design crowdsourcing contests: Managing participation, contribution appropriateness, and winning trade-offs," Technological Forecasting and Social Change, Elsevier, vol. 191(C).
    35. Ho Cheung Brian Lee & Sulin Ba & Xinxin Li & Jan Stallaert, 2018. "Salience Bias in Crowdsourcing Contests," Information Systems Research, INFORMS, vol. 29(2), pages 401-418, June.
    36. Liran Einav, 2005. "Informational Asymmetries and Observational Learning in Search," Journal of Risk and Uncertainty, Springer, vol. 30(3), pages 241-259, May.
    37. Barry J. Nalebuff & Joseph E. Stiglitz, 1983. "Prices and Incentives: Towards a General Theory of Compensation and Competition," Bell Journal of Economics, The RAND Corporation, vol. 14(1), pages 21-43, Spring.
    38. Kevin J. Boudreau & Nicola Lacetera & Karim R. Lakhani, 2011. "Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis," Management Science, INFORMS, vol. 57(5), pages 843-863, May.
    39. Jeffrey A. Roberts & Il-Horn Hann & Sandra A. Slaughter, 2006. "Understanding the Motivations, Participation, and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects," Management Science, INFORMS, vol. 52(7), pages 984-999, July.
    40. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    41. O'Keeffe, Mary & Viscusi, W Kip & Zeckhauser, Richard J, 1984. "Economic Contests: Comparative Reward Schemes," Journal of Labor Economics, University of Chicago Press, vol. 2(1), pages 27-56, January.
    42. Ronald Harstad, 2000. "Dominant Strategy Adoption and Bidders' Experience with Pricing Rules," Experimental Economics, Springer;Economic Science Association, vol. 3(3), pages 261-280, December.
    43. Thomas Giebe & Radosveta Ivanova-Stenzel & Martin G. Kocher & Simeon Schudy, 2024. "Cross-game learning and cognitive ability in auctions," Experimental Economics, Springer;Economic Science Association, vol. 27(1), pages 80-108, March.
    44. Ajay Kalra & Mengze Shi, 2001. "Designing Optimal Sales Contests: A Theoretical Perspective," Marketing Science, INFORMS, vol. 20(2), pages 170-193, December.
    45. Powell, Melanie & Ansic, David, 1997. "Gender differences in risk behaviour in financial decision-making: An experimental analysis," Journal of Economic Psychology, Elsevier, vol. 18(6), pages 605-628, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    2. Roman M. Sheremeta, 2016. "The pros and cons of workplace tournaments," World of Labour, LISER, pages 302-302, October.
    3. Curtis R. Price & Roman M. Sheremeta, 2015. "Endowment Origin, Demographic Effects, and Individual Preferences in Contests," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 24(3), pages 597-619, September.
    4. Hendrik Sonnabend, 2020. "On discouraging environments in team contests: Evidence from top‐level beach volleyball," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(6), pages 986-997, September.
    5. Llorente-Saguer, Aniol & Sheremeta, Roman M. & Szech, Nora, 2023. "Designing contests between heterogeneous contestants: An experimental study of tie-breaks and bid-caps in all-pay auctions," European Economic Review, Elsevier, vol. 154(C).
    6. Llorente-Saguer, Aniol & Sheremeta, Roman & Szech, Nora, 2016. "Designing Contests Between Heterogeneous Contestants: An Experimental Study of Tie-Breaks and Bid-Caps in All-Pay Auctions," MPRA Paper 71202, University Library of Munich, Germany.
    7. Graff, Frederik & Grund, Christian & Harbring, Christine, 2021. "Competing on the Holodeck - The effect of virtual peers and heterogeneity in dynamic tournaments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 90(C).
    8. Cason, Timothy N. & Masters, William A. & Sheremeta, Roman M., 2020. "Winner-take-all and proportional-prize contests: Theory and experimental results," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 314-327.
    9. Kimbrough, Erik O. & Laughren, Kevin & Sheremeta, Roman, 2020. "War and conflict in economics: Theories, applications, and recent trends," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 998-1013.
    10. Roman M. Sheremeta, 2013. "Overbidding And Heterogeneous Behavior In Contest Experiments," Journal of Economic Surveys, Wiley Blackwell, vol. 27(3), pages 491-514, July.
    11. Jörg Claussen & Maria A. Halbinger & Lior Zalmanson, 2026. "Challenge Accepted: The Effects of Contest Participation on a User-Generated Content Community," Manufacturing & Service Operations Management, INFORMS, vol. 28(2), pages 421-439, March.
    12. Philip Brookins & Dmitry Ryvkin, 2014. "An experimental study of bidding in contests of incomplete information," Experimental Economics, Springer;Economic Science Association, vol. 17(2), pages 245-261, June.
    13. Roman Sheremeta, 2018. "Experimental Research on Contests," Working Papers 18-07, Chapman University, Economic Science Institute.
    14. Emmanuel Dechenaux & Shakun D. Mago, 2023. "Contests with revisions," Experimental Economics, Springer;Economic Science Association, vol. 26(4), pages 915-954, September.
    15. Jens Forderer & Gordon Burtch, 2025. "Estimating Career Benefits from Online Community Leadership: Evidence from Stack Exchange Moderators," Management Science, INFORMS, vol. 71(3), pages 2443-2466, March.
    16. Enzo Brox & Daniel Goller, 2026. "Tournaments, Contestant Heterogeneity, and Performance," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 4(2), pages 583-619.
    17. Necker, Sarah & Paetzel, Fabian, 2023. "The effect of losing and winning on cheating and effort in repeated competitions," Journal of Economic Psychology, Elsevier, vol. 98(C).
    18. Brookins, Philip & Jindapon, Paan, 2021. "Risk preference heterogeneity in group contests," Journal of Mathematical Economics, Elsevier, vol. 95(C).
    19. Bhattacharya, Puja, 2016. "Inter-team contests with power differential," Journal of Economic Behavior & Organization, Elsevier, vol. 132(PA), pages 157-175.
    20. Hoffmann, Magnus & Kolmar, Martin, 2017. "Distributional preferences in probabilistic and share contests," Journal of Economic Behavior & Organization, Elsevier, vol. 142(C), pages 120-139.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orstsc:v:10:y:2025:i:4:p:421-444. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.