IDEAS home Printed from https://ideas.repec.org/a/inm/ormoor/v38y2013i1p92-107.html
   My bibliography  Save this article

Bandit Problems with Lévy Processes

Author

Listed:
  • Asaf Cohen

    () (The School of Mathematical Sciences, Tel Aviv University, Tel Aviv 69978, Israel)

  • Eilon Solan

    () (The School of Mathematical Sciences, Tel Aviv University, Tel Aviv 69978, Israel)

Abstract

Bandit problems model the trade-off between exploration and exploitation in various decision problems. We study two-armed bandit problems in continuous time, where the risky arm can have two types: High or Low; both types yield stochastic payoffs generated by a Lévy process. We show that the optimal strategy is a cut-off strategy and we provide an explicit expression for the cut-off and for the optimal payoff.

Suggested Citation

  • Asaf Cohen & Eilon Solan, 2013. "Bandit Problems with Lévy Processes," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 92-107, February.
    • Handle: RePEc:inm:ormoor:v:38:y:2013:i:1:p:92-107
    DOI: 10.1287/moor.1120.0564
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/moor.1120.0564
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nicolas Klein & Sven Rady, 2011. "Negatively Correlated Bandits," Review of Economic Studies, Oxford University Press, vol. 78(2), pages 693-732.
    2. Roberts, Kevin & Weitzman, Martin L, 1981. "Funding Criteria for Research, Development, and Exploration Projects," Econometrica, Econometric Society, vol. 49(5), pages 1261-1288, September.
    3. Godfrey Keller & Sven Rady & Martin Cripps, 2005. "Strategic Experimentation with Exponential Bandits," Econometrica, Econometric Society, vol. 73(1), pages 39-68, January.
    4. Rothschild, Michael, 1974. "A two-armed bandit theory of market pricing," Journal of Economic Theory, Elsevier, vol. 9(2), pages 185-202, October.
    5. Keller, Godfrey & Rady, Sven, 2010. "Strategic experimentation with Poisson bandits," Theoretical Economics, Econometric Society, vol. 5(2), May.
    6. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    7. Savas Dayanik & Semih Onur Sezer, 2006. "Compound Poisson Disorder Problem," Mathematics of Operations Research, INFORMS, vol. 31(4), pages 649-672, November.
    8. Moscarini, Giuseppe & Squintani, Francesco, 2010. "Competitive experimentation with private information: The survivor's curse," Journal of Economic Theory, Elsevier, vol. 145(2), pages 639-660, March.
    9. Gapeev, Pavel V., 2005. "The disorder problem for compound Poisson processes with exponential jumps," LSE Research Online Documents on Economics 3219, London School of Economics and Political Science, LSE Library.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Keller, Godfrey & Rady, Sven, 2020. "Undiscounted bandit games," Games and Economic Behavior, Elsevier, vol. 124(C), pages 43-61.
    2. Roland Fryer & Philipp Harms, 2018. "Two-Armed Restless Bandits with Imperfect Information: Stochastic Control and Indexability," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 399-427, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Keller, Godfrey & Novák, Vladimír & Willems, Tim, 2019. "A note on optimal experimentation under risk aversion," Journal of Economic Theory, Elsevier, vol. 179(C), pages 476-487.
    2. Roland G. Fryer, Jr. & Philipp Harms, 2013. "Two-Armed Restless Bandits with Imperfect Information: Stochastic Control and Indexability," NBER Working Papers 19043, National Bureau of Economic Research, Inc.
    3. Heidhues, Paul & Rady, Sven & Strack, Philipp, 2015. "Strategic experimentation with private payoffs," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 531-551.
    4. Rosenberg, Dinah & Salomon, Antoine & Vieille, Nicolas, 2013. "On games of strategic experimentation," Games and Economic Behavior, Elsevier, vol. 82(C), pages 31-51.
    5. Deimen, Inga & Wirtz, Julia, 2016. "A Bandit Model of Two-Dimensional Uncertainty -- Rationalizing Mindsets," VfS Annual Conference 2016 (Augsburg): Demographic Change 145931, Verein für Socialpolitik / German Economic Association.
    6. Ganglmair, Bernhard & Simcoe, Timothy & Tarantino, Emanuele, 2018. "Learning When to Quit: An Empirical Model of Experimentation," CEPR Discussion Papers 12733, C.E.P.R. Discussion Papers.
    7. Klein, Nicolas, 2013. "Strategic learning in teams," Games and Economic Behavior, Elsevier, vol. 82(C), pages 636-657.
    8. Agbo, Maxime, 2015. "A perpetual search for talents across overlapping generations: A learning process," Mathematical Social Sciences, Elsevier, vol. 76(C), pages 131-145.
    9. Roland Fryer & Philipp Harms, 2018. "Two-Armed Restless Bandits with Imperfect Information: Stochastic Control and Indexability," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 399-427, May.
    10. Hoppe-Wewetzer, Heidrun C. & Katsenos, Georgios & Ozdenoren, Emre, 2019. "Experimentation, Learning, and Preemption," CEPR Discussion Papers 13483, C.E.P.R. Discussion Papers.
    11. Kaustav Das, 2014. "Strategic Experimentation with Competition and Private Arrival of Information," Discussion Papers 1404, University of Exeter, Department of Economics.
    12. Keller, Godfrey & Rady, Sven, 2010. "Strategic experimentation with Poisson bandits," Theoretical Economics, Econometric Society, vol. 5(2), May.
    13. Besanko, David & Tong, Jian & Wu, Jianjun, 2016. "Subsidizing research programs with "if" and "when" uncertainty in the face of severe informational constraints," Discussion Paper Series In Economics And Econometrics 1605, Economics Division, School of Social Sciences, University of Southampton.
    14. Inga Deimen & Julia Wirtz, 2021. "Control, Cost, and Confidence:Perseverance and Procrastination in the Face of Failure," Bristol Economics Discussion Papers 21/738, School of Economics, University of Bristol, UK.
    15. Sorensen, Morten, 2007. "Learning by Investing: Evidence from Venture Capital," SIFR Research Report Series 53, Institute for Financial Research.
    16. Shaun Larcom & Ferdinand Rauch & Tim Willems, 2017. "The Benefits of Forced Experimentation: Striking Evidence from the London Underground Network," The Quarterly Journal of Economics, Oxford University Press, vol. 132(4), pages 2019-2055.
    17. Keller, Godfrey & Rady, Sven, 2020. "Undiscounted bandit games," Games and Economic Behavior, Elsevier, vol. 124(C), pages 43-61.
    18. Forand, Jean Guillaume, 2015. "Keeping your options open," Journal of Economic Dynamics and Control, Elsevier, vol. 53(C), pages 47-68.
    19. Xie, Yinxi & Xie, Yang, 2017. "Machiavellian experimentation," Journal of Comparative Economics, Elsevier, vol. 45(4), pages 685-711.
    20. Kaustav Das & Nicolas Klein & Katharina Schmid, 2020. "Strategic experimentation with asymmetric players," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 69(4), pages 1147-1175, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormoor:v:38:y:2013:i:1:p:92-107. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Matthew Walls). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.