IDEAS home Printed from https://ideas.repec.org/p/ess/wpaper/id12569.html
   My bibliography  Save this paper

Learning When to Quit: An Empirical Model of Experimentation

Author

Listed:
  • Bernhard Ganglmair

    ()

  • Timothy Simcoe

    ()

  • Emanuele Tarantino

    ()

Abstract

The paper studies a dynamic model of the decision to continue or abandon a research project. Researchers improve their ideas over time and also learn whether those ideas will be adopted by the scientific community. Projects are abandoned as researchers grow more pessimistic about their chance of success. It estimates the structural parameters of this dynamic decision problem using a novel data set that contains information on both successful and abandoned projects submitted to the Internet Engineering Task Force (IETF), an organization that creates and maintains internet standards. Using the model and parameter estimates, it also simulates two counterfactual policies: a cost-subsidy and a prize-based incentive scheme. For a fixed budget, subsidies have a larger impact on research output, but prizes perform better when accounting for researchers’ opportunity costs.

Suggested Citation

  • Bernhard Ganglmair & Timothy Simcoe & Emanuele Tarantino, 2018. "Learning When to Quit: An Empirical Model of Experimentation," Working Papers id:12569, eSocialSciences.
  • Handle: RePEc:ess:wpaper:id:12569
    Note: Institutional Papers
    as

    Download full text from publisher

    File URL: http://www.esocialsciences.org/Download/repecDownload.aspx?fname=A20183615316_29.pdf&fcategory=Articles&AId=12569&fref=repec
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," Review of Economic Studies, Oxford University Press, vol. 74(4), pages 1059-1087.
    2. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    3. Roberts, Kevin & Weitzman, Martin L, 1981. "Funding Criteria for Research, Development, and Exploration Projects," Econometrica, Econometric Society, vol. 49(5), pages 1261-1288, September.
    4. Nicholas Bloom & Luis Garicano & Raffaella Sadun & John Van Reenen, 2014. "The Distinct Effects of Information Technology and Communication Technology on Firm Organization," Management Science, INFORMS, vol. 60(12), pages 2859-2885, December.
    5. Barton H. Hamilton & Jack A. Nickerson & Hideo Owan, 2003. "Team Incentives and Worker Heterogeneity: An Empirical Analysis of the Impact of Teams on Productivity and Participation," Journal of Political Economy, University of Chicago Press, vol. 111(3), pages 465-497, June.
    6. Godfrey Keller & Sven Rady & Martin Cripps, 2005. "Strategic Experimentation with Exponential Bandits," Econometrica, Econometric Society, vol. 73(1), pages 39-68, January.
    7. Timothy Simcoe, 2012. "Standard Setting Committees: Consensus Governance for Shared Technology Platforms," American Economic Review, American Economic Association, vol. 102(1), pages 305-336, February.
    8. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    9. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.
    10. Tülin Erdem & Michael P. Keane, 1996. "Decision-Making Under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets," Marketing Science, INFORMS, vol. 15(1), pages 1-20.
    11. Moscarini, Giuseppe & Squintani, Francesco, 2010. "Competitive experimentation with private information: The survivor's curse," Journal of Economic Theory, Elsevier, vol. 145(2), pages 639-660, March.
    12. Murray, Fiona & Stern, Scott & Campbell, Georgina & MacCormack, Alan, 2012. "Grand Innovation Prizes: A theoretical, normative, and empirical evaluation," Research Policy, Elsevier, vol. 41(10), pages 1779-1792.
    13. Hugo Hopenhayn & Gerard Llobet & Matthew Mitchell, 2006. "Rewarding Sequential Innovators: Prizes, Patents, and Buyouts," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 1041-1068, December.
    14. Thierry Magnac & David Thesmar, 2002. "Identifying Dynamic Discrete Decision Processes," Econometrica, Econometric Society, vol. 70(2), pages 801-816, March.
    15. Gregory S. Crawford & Matthew Shum, 2005. "Uncertainty and Learning in Pharmaceutical Demand," Econometrica, Econometric Society, vol. 73(4), pages 1137-1173, July.
    16. Patrick Bolton & Christopher Harris, 1999. "Strategic Experimentation," Econometrica, Econometric Society, vol. 67(2), pages 349-374, March.
    17. Robert L. Bray & Decio Coviello & Andrea Ichino & Nicola Persico, 2016. "Multitasking, Multiarmed Bandits, and the Italian Judiciary," Manufacturing & Service Operations Management, INFORMS, vol. 18(4), pages 545-558, October.
    18. Galasso, Alberto & Mitchell, Matthew & Virag, Gabor, 2016. "Market outcomes and dynamic patent buyouts," International Journal of Industrial Organization, Elsevier, vol. 48(C), pages 207-243.
    19. Nicolas Serrano-Velarde & Douglas Hanley & Ufuk Akcigit, 2012. "Back to Basics: Basic Research Spillovers, Innovation Policy and Growth," 2012 Meeting Papers 665, Society for Economic Dynamics.
    20. David Hirshleifer & Angie Low & Siew Hong Teoh, 2012. "Are Overconfident CEOs Better Innovators?," Journal of Finance, American Finance Association, vol. 67(4), pages 1457-1498, August.
    21. Tat Y. Chan & Barton H. Hamilton, 2006. "Learning, Private Information, and the Economic Evaluation of Randomized Experiments," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 997-1040, December.
    22. Ufuk Akcigit & Qingmin Liu, 2016. "The Role Of Information In Innovation And Competition," Journal of the European Economic Association, European Economic Association, vol. 14(4), pages 828-870, August.
    23. Pakes, Ariel S, 1986. "Patents as Options: Some Estimates of the Value of Holding European Patent Stocks," Econometrica, Econometric Society, vol. 54(4), pages 755-784, July.
    24. Glenn Ellison, 2002. "Evolving Standards for Academic Publishing: A q-r Theory," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 994-1034, October.
    25. Heidhues, Paul & Rady, Sven & Strack, Philipp, 2015. "Strategic experimentation with private payoffs," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 531-551.
    26. William R. Kerr & Ramana Nanda & Matthew Rhodes-Kropf, 2014. "Entrepreneurship as Experimentation," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 25-48, Summer.
    27. Daniel P. Gross, 2017. "Performance feedback in competitive product development," RAND Journal of Economics, RAND Corporation, vol. 48(2), pages 438-466, May.
    28. Richard Ericson & Ariel Pakes, 1995. "Markov-Perfect Industry Dynamics: A Framework for Empirical Work," Review of Economic Studies, Oxford University Press, vol. 62(1), pages 53-82.
    29. Richard Gilbert & Carl Shapiro, 1990. "Optimal Patent Length and Breadth," RAND Journal of Economics, The RAND Corporation, vol. 21(1), pages 106-112, Spring.
    30. Alberto Galasso & Timothy S. Simcoe, 2011. "CEO Overconfidence and Innovation," Management Science, INFORMS, vol. 57(8), pages 1469-1484, August.
    31. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Learning Models: An Assessment of Progress, Challenges and New Developments," Economics Papers 2013-W07, Economics Group, Nuffield College, University of Oxford.
    32. Giuseppe Moscarini & Lones Smith, 2001. "The Optimal Level of Experimentation," Econometrica, Econometric Society, vol. 69(6), pages 1629-1644, November.
    33. Sabrina T. Howell, 2017. "Reducing Information Frictions in Venture Capital: The Role of New Venture Competitions," NBER Working Papers 23874, National Bureau of Economic Research, Inc.
    34. Bernhard Ganglmair & Emanuele Tarantino, 2014. "Conversation with secrets," RAND Journal of Economics, RAND Corporation, vol. 45(2), pages 273-302, June.
    35. Michael Roach & Wesley M. Cohen, 2013. "Lens or Prism? Patent Citations as a Measure of Knowledge Flows from Public Research," Management Science, INFORMS, vol. 59(2), pages 504-525, October.
    36. Berend Wierenga & Ralf van der Lans (ed.), 2017. "Handbook of Marketing Decision Models," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-56941-3, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Baron, Justus, 2020. "Counting standard contributions to measure the value of patent portfolios - A tale of apples and oranges," Telecommunications Policy, Elsevier, vol. 44(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Keller, Godfrey & Novák, Vladimír & Willems, Tim, 2019. "A note on optimal experimentation under risk aversion," Journal of Economic Theory, Elsevier, vol. 179(C), pages 476-487.
    2. Kohei Kawaguchi, 2021. "When Will Workers Follow an Algorithm? A Field Experiment with a Retail Business," Management Science, INFORMS, vol. 67(3), pages 1670-1695, March.
    3. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    4. S. Sriram & Pradeep K. Chintagunta & Puneet Manchanda, 2015. "Service Quality Variability and Termination Behavior," Management Science, INFORMS, vol. 61(11), pages 2739-2759, November.
    5. Sorensen, Morten, 2007. "Learning by Investing: Evidence from Venture Capital," SIFR Research Report Series 53, Institute for Financial Research.
    6. Jason R. Blevins & Ahmed Khwaja & Nathan Yang, 2018. "Firm Expansion, Size Spillovers, and Market Dominance in Retail Chain Dynamics," Management Science, INFORMS, vol. 64(9), pages 4070-4093.
    7. Sabrina T. Howell, 2017. "Reducing Information Frictions in Venture Capital: The Role of New Venture Competitions," NBER Working Papers 23874, National Bureau of Economic Research, Inc.
    8. Ke, T. Tony & Villas-Boas, J. Miguel, 2019. "Optimal learning before choice," Journal of Economic Theory, Elsevier, vol. 180(C), pages 383-437.
    9. Gomes, Renato & Gottlieb, Daniel & Maestri, Lucas, 2016. "Experimentation and project selection: Screening and learning," Games and Economic Behavior, Elsevier, vol. 96(C), pages 145-169.
    10. Ching, Andrew T. & Erdem, Tülin & Keane, Michael P., 2014. "A simple method to estimate the roles of learning, inventories and category consideration in consumer choice," Journal of choice modelling, Elsevier, vol. 13(C), pages 60-72.
    11. Asaf Cohen & Eilon Solan, 2013. "Bandit Problems with Lévy Processes," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 92-107, February.
    12. Hu, Yingyao & Kayaba, Yutaka & Shum, Matthew, 2013. "Nonparametric learning rules from bandit experiments: The eyes have it!," Games and Economic Behavior, Elsevier, vol. 81(C), pages 215-231.
    13. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2020. "How much do consumers know about the quality of products? Evidence from the diaper market," The Japanese Economic Review, Springer, vol. 71(4), pages 541-569, October.
    14. Jürgen Maurer & Katherine M. Harris, 2016. "Learning to Trust Flu Shots: Quasi‐Experimental Evidence from the 2009 Swine Flu Pandemic," Health Economics, John Wiley & Sons, Ltd., vol. 25(9), pages 1148-1162, September.
    15. Ewens, Michael & Nanda, Ramana & Rhodes-Kropf, Matthew, 2018. "Cost of experimentation and the evolution of venture capital," Journal of Financial Economics, Elsevier, vol. 128(3), pages 422-442.
    16. T. Tony Ke & Zuo-Jun Max Shen & J. Miguel Villas-Boas, 2016. "Search for Information on Multiple Products," Management Science, INFORMS, vol. 62(12), pages 3576-3603, December.
    17. Marcoul, Philippe & Weninger, Quinn, 2008. "Search and active learning with correlated information: Empirical evidence from mid-Atlantic clam fishermen," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1921-1948, June.
    18. Andrew T. Ching & Hyunwoo Lim, 2020. "A Structural Model of Correlated Learning and Late-Mover Advantages: The Case of Statins," Management Science, INFORMS, vol. 66(3), pages 1095-1123, March.
    19. A. Ronald Gallant & Han Hong & Ahmed Khwaja, 2018. "The Dynamic Spillovers of Entry: An Application to the Generic Drug Industry," Management Science, INFORMS, vol. 64(3), pages 1189-1211, March.
    20. Hai Che & Tülin Erdem & T. Sabri Öncü, 2015. "Consumer learning and evolution of consumer brand preferences," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 173-202, September.

    More about this item

    Keywords

    eSS; learning; Internet Engineering Task Force (IETF); internet standard; parameters; counterfactual policies; cost-subsidy; prize-based incentive scheme; decision; research report; scientific community; novel data.;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ess:wpaper:id:12569. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Padma Prakash). General contact details of provider: http://www.esocialsciences.org .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.