IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v68y2022i7p4858-4877.html
   My bibliography  Save this article

The Role of Feedback in Dynamic Crowdsourcing Contests: A Structural Empirical Analysis

Author

Listed:
  • Zhaohui (Zoey) Jiang

    (Tepper School of Business, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213)

  • Yan Huang

    (Tepper School of Business, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213)

  • Damian R. Beil

    (Stephen M. Ross School of Business, University of Michigan, Ann Arbor, Michigan, 48109)

Abstract

In this paper, we empirically examine the impact of performance feedback on the outcome of crowdsourcing contests. We develop a dynamic structural model to capture the economic processes that drive contest participants’ behavior and estimate the model using a detailed data set about real online logo design contests. Our rich model captures key features of the crowdsourcing context, including a large participant pool; entries by new participants throughout the contest; exploitation (revision of previous submissions) and exploration (radically novel submissions) behaviors by contest incumbents; and the participants’ strategic choice among these entry, exploration, and exploitation decisions in a dynamic game. Using counterfactual simulations, we compare the outcome of crowdsourcing contests under alternative feedback disclosure policies and award levels. Our simulation results suggest that, despite its prevalence on many platforms, the full feedback policy (providing feedback throughout the contest) may not be optimal. The late feedback policy (providing feedback only in the second half of the contest) leads to a better overall contest outcome.

Suggested Citation

  • Zhaohui (Zoey) Jiang & Yan Huang & Damian R. Beil, 2022. "The Role of Feedback in Dynamic Crowdsourcing Contests: A Structural Empirical Analysis," Management Science, INFORMS, vol. 68(7), pages 4858-4877, July.
    • Handle: RePEc:inm:ormnsc:v:68:y:2022:i:7:p:4858-4877
    DOI: 10.1287/mnsc.2021.4140
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.2021.4140
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.2021.4140?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
    2. Benny Moldovanu & Aner Sela, 2008. "The Optimal Allocation of Prizes in Contests," Springer Books, in: Roger D. Congleton & Arye L. Hillman & Kai A. Konrad (ed.), 40 Years of Research on Rent Seeking 1, pages 615-631, Springer.
    3. Kevin J. Boudreau & Karim R. Lakhani & Michael Menietti, 2016. "Performance responses to competition across skill levels in rank-order tournaments: field evidence and implications for tournament design," RAND Journal of Economics, RAND Corporation, vol. 47(1), pages 140-165, February.
    4. Sanjiv Erat & Vish Krishnan, 2012. "Managing Delegated Search Over Design Spaces," Management Science, INFORMS, vol. 58(3), pages 606-623, March.
    5. Patrick Bajari & Victor Chernozhukov & Han Hong & Denis Nekipelov, 2015. "Identification and Efficient Semiparametric Estimation of a Dynamic Discrete Game," NBER Working Papers 21125, National Bureau of Economic Research, Inc.
    6. V. Joseph Hotz & Robert A. Miller, 1993. "Conditional Choice Probabilities and the Estimation of Dynamic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 60(3), pages 497-529.
    7. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    8. Mitsuru Igami, 2017. "Estimating the Innovator’s Dilemma: Structural Analysis of Creative Destruction in the Hard Disk Drive Industry, 1981–1998," Journal of Political Economy, University of Chicago Press, vol. 125(3), pages 798-847.
    9. Keane, Michael P & Wolpin, Kenneth I, 1994. "The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 76(4), pages 648-672, November.
    10. Jürgen Mihm & Jochen Schlapp, 2019. "Sourcing Innovation: On Feedback in Contests," Management Science, INFORMS, vol. 65(2), pages 559-576, February.
    11. Christian Terwiesch & Yi Xu, 2008. "Innovation Contests, Open Innovation, and Multiagent Problem Solving," Management Science, INFORMS, vol. 54(9), pages 1529-1543, September.
    12. Ashish Kabra & Elena Belavina & Karan Girotra, 2020. "Bike-Share Systems: Accessibility and Availability," Management Science, INFORMS, vol. 66(9), pages 3803-3824, September.
    13. Kevin J. Boudreau & Nicola Lacetera & Karim R. Lakhani, 2011. "Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis," Management Science, INFORMS, vol. 57(5), pages 843-863, May.
    14. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    15. Aoyagi, Masaki, 2010. "Information feedback in a dynamic tournament," Games and Economic Behavior, Elsevier, vol. 70(2), pages 242-260, November.
    16. Laura J. Kornish & Karl T. Ulrich, 2011. "Opportunity Spaces in Innovation: Empirical Analysis of Large Samples of Ideas," Management Science, INFORMS, vol. 57(1), pages 107-128, January.
    17. Pu He & Fanyin Zheng & Elena Belavina & Karan Girotra, 2021. "Customer Preference and Station Network in the London Bike-Share System," Management Science, INFORMS, vol. 67(3), pages 1392-1412, March.
    18. Paul R. Milgrom & Robert J. Weber, 1985. "Distributional Strategies for Games with Incomplete Information," Mathematics of Operations Research, INFORMS, vol. 10(4), pages 619-632, November.
    19. Zeynep Akşin & Barış Ata & Seyed Morteza Emadi & Che-Lin Su, 2013. "Structural Estimation of Callers' Delay Sensitivity in Call Centers," Management Science, INFORMS, vol. 59(12), pages 2727-2746, December.
    20. Marina Halac & Navin Kartik & Qingmin Liu, 2017. "Contests for Experimentation," Journal of Political Economy, University of Chicago Press, vol. 125(5), pages 1523-1569.
    21. Daniel P. Gross, 2017. "Performance feedback in competitive product development," RAND Journal of Economics, RAND Corporation, vol. 48(2), pages 438-466, May.
    22. Fudenberg, Drew & Gilbert, Richard & Stiglitz, Joseph & Tirole, Jean, 1983. "Preemption, leapfrogging and competition in patent races," European Economic Review, Elsevier, vol. 22(1), pages 3-31, June.
    23. Andrew Sweeting, 2009. "The strategic timing incentives of commercial radio stations: An empirical analysis using multiple equilibria," RAND Journal of Economics, RAND Corporation, vol. 40(4), pages 710-742, December.
    24. Maria Goltsman & Arijit Mukherjee, 2011. "Interim Performance Feedback in Multistage Tournaments: The Optimality of Partial Disclosure," Journal of Labor Economics, University of Chicago Press, vol. 29(2), pages 229-265.
    25. Kostas Bimpikis & Wedad J. Elmaghraby & Ken Moon & Wenchang Zhang, 2020. "Managing Market Thickness in Online Business-to-Business Markets," Management Science, INFORMS, vol. 66(12), pages 5783-5822, December.
    26. Karan Girotra & Christian Terwiesch & Karl T. Ulrich, 2010. "Idea Generation and the Quality of the Best Idea," Management Science, INFORMS, vol. 56(4), pages 591-605, April.
    27. Kostas Bimpikis & Shayan Ehsani & Mohamed Mostagir, 2019. "Designing Dynamic Contests," Operations Research, INFORMS, vol. 67(2), pages 339-356, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yixin (Iris) Wang & Jun Li & Ravi Anupindi, 2023. "Manufacturing and Regulatory Barriers to Generic Drug Competition: A Structural Model Approach," Management Science, INFORMS, vol. 69(3), pages 1449-1467, March.
    2. Emmanuel Dechenaux & Shakun D. Mago, 2023. "Contests with revisions," Experimental Economics, Springer;Economic Science Association, vol. 26(4), pages 915-954, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jürgen Mihm & Jochen Schlapp, 2019. "Sourcing Innovation: On Feedback in Contests," Management Science, INFORMS, vol. 65(2), pages 559-576, February.
    2. Ming Hu & Lu Wang, 2021. "Joint vs. Separate Crowdsourcing Contests," Management Science, INFORMS, vol. 67(5), pages 2711-2728, May.
    3. Pavel Kireyev, 2016. "Markets for Ideas: Prize Structure, Entry Limits, and the Design of Ideation Contests," Harvard Business School Working Papers 16-129, Harvard Business School.
    4. Laura J. Kornish & Jeremy Hutchison‐Krupat, 2017. "Research on Idea Generation and Selection: Implications for Management of Technology," Production and Operations Management, Production and Operations Management Society, vol. 26(4), pages 633-651, April.
    5. Joel O. Wooten, 2022. "Leaps in innovation and the Bannister effect in contests," Production and Operations Management, Production and Operations Management Society, vol. 31(6), pages 2646-2663, June.
    6. Segev, Ella, 2020. "Crowdsourcing contests," European Journal of Operational Research, Elsevier, vol. 281(2), pages 241-255.
    7. Cédric Gutierrez & Tomasz Obloj & Douglas H. Frank, 2021. "Better to have led and lost than never to have led at all? Lost leadership and effort provision in dynamic tournaments," Strategic Management Journal, Wiley Blackwell, vol. 42(4), pages 774-801, April.
    8. Dmitry Ryvkin, 2022. "To Fight or to Give Up? Dynamic Contests with a Deadline," Management Science, INFORMS, vol. 68(11), pages 8144-8165, November.
    9. Laurence Ales & Soo‐Haeng Cho & Ersin Körpeoğlu, 2021. "Innovation Tournaments with Multiple Contributors," Production and Operations Management, Production and Operations Management Society, vol. 30(6), pages 1772-1784, June.
    10. Daniel P. Gross, 2017. "Performance feedback in competitive product development," RAND Journal of Economics, RAND Corporation, vol. 48(2), pages 438-466, May.
    11. Letina, Igor & Benkert, Jean-Michel, 2016. "Designing Dynamic Research Tournaments," VfS Annual Conference 2016 (Augsburg): Demographic Change 145738, Verein für Socialpolitik / German Economic Association.
    12. Ying-Ju Chen & Tinglong Dai & C. Gizem Korpeoglu & Ersin Körpeoğlu & Ozge Sahin & Christopher S. Tang & Shihong Xiao, 2020. "OM Forum—Innovative Online Platforms: Research Opportunities," Manufacturing & Service Operations Management, INFORMS, vol. 22(3), pages 430-445, May.
    13. Jean-Michel Benkert & Igor Letina, 2020. "Designing Dynamic Research Contests," American Economic Journal: Microeconomics, American Economic Association, vol. 12(4), pages 270-289, November.
    14. Pavel Kireyev, 2020. "Markets for ideas: prize structure, entry limits, and the design of ideation contests," RAND Journal of Economics, RAND Corporation, vol. 51(2), pages 563-588, June.
    15. Lakshminarayana Nittala & Sanjiv Erat & Vish Krishnan, 2022. "Designing internal innovation contests," Production and Operations Management, Production and Operations Management Society, vol. 31(5), pages 1963-1976, May.
    16. Jesse Bockstedt & Cheryl Druehl & Anant Mishra, 2022. "Incentives and Stars: Competition in Innovation Contests with Participant and Submission Visibility," Production and Operations Management, Production and Operations Management Society, vol. 31(3), pages 1372-1393, March.
    17. Bo Chen & Bo Chen & Dmitriy Knyazev, 2022. "Information disclosure in dynamic research contests," RAND Journal of Economics, RAND Corporation, vol. 53(1), pages 113-137, March.
    18. Daniel P. Gross, 2020. "Creativity Under Fire: The Effects of Competition on Creative Production," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 583-599, July.
    19. Hossain, Mokter, 2018. "Motivations, challenges, and opportunities of successful solvers on an innovation intermediary platform," Technological Forecasting and Social Change, Elsevier, vol. 128(C), pages 67-73.
    20. Llorente-Saguer, Aniol & Sheremeta, Roman M. & Szech, Nora, 2023. "Designing contests between heterogeneous contestants: An experimental study of tie-breaks and bid-caps in all-pay auctions," European Economic Review, Elsevier, vol. 154(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:68:y:2022:i:7:p:4858-4877. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.