IDEAS home Printed from https://ideas.repec.org/a/spr/dyngam/v7y2017i2d10.1007_s13235-016-0186-2.html
   My bibliography  Save this article

Sporadic Overtaking Optimality in Markov Decision Problems

Author

Listed:
  • János Flesch

    (Maastricht University)

  • Arkadi Predtetchinski

    (Maastricht University)

  • Eilon Solan

    (Tel-Aviv University)

Abstract

This paper examines a notion of sporadic overtaking optimality in the context of Markov decision problems (MDP). For the class of deterministic MDPs, we prove the existence of pure stationary sporadic overtaking optimal strategies under both the discounted and the average payoff evaluations. Moreover, we examine logical connections between sporadic overtaking optimality and Blackwell optimality. In the class of nondeterministic MDPs, we give examples that admit no sporadic overtaking optimal strategy and discuss a number of alternative definitions of this concept.

Suggested Citation

  • János Flesch & Arkadi Predtetchinski & Eilon Solan, 2017. "Sporadic Overtaking Optimality in Markov Decision Problems," Dynamic Games and Applications, Springer, vol. 7(2), pages 212-228, June.
    • Handle: RePEc:spr:dyngam:v:7:y:2017:i:2:d:10.1007_s13235-016-0186-2
    DOI: 10.1007/s13235-016-0186-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s13235-016-0186-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s13235-016-0186-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alexander J. Zaslavski, 2014. "Turnpike Phenomenon and Infinite Horizon Optimal Control," Springer Optimization and Its Applications, Springer, edition 127, number 978-3-319-08828-0, September.
    2. Méder, Z.Z. & Flesch, J. & Peeters, R.J.A.P., 2012. "Optimal choice for finite and infinite horizons," Research Memorandum 024, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    3. Arie Leizarowitz, 1996. "Overtaking and Almost-Sure Optimality for Infinite Horizon Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 21(1), pages 158-181, February.
    4. Andrzej S. Nowak & Oscar Vega-Amaya, 1999. "A counterexample on overtaking optimality," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 49(3), pages 435-439, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Galit Ashkenazi-Golan & János Flesch & Arkadi Predtetchinski & Eilon Solan, 2020. "Reachability and Safety Objectives in Markov Decision Processes on Long but Finite Horizons," Journal of Optimization Theory and Applications, Springer, vol. 185(3), pages 945-965, June.
    2. Adam Jonsson, 2023. "An axiomatic approach to Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 97(1), pages 117-133, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Galit Ashkenazi-Golan & János Flesch & Arkadi Predtetchinski & Eilon Solan, 2020. "Reachability and Safety Objectives in Markov Decision Processes on Long but Finite Horizons," Journal of Optimization Theory and Applications, Springer, vol. 185(3), pages 945-965, June.
    2. Francesco Bartaloni, 2021. "Existence of the Optimum in Shallow Lake Type Models with Hysteresis Effect," Journal of Optimization Theory and Applications, Springer, vol. 190(2), pages 358-392, August.
    3. Grüne, Lars & Semmler, Willi & Stieler, Marleen, 2015. "Using nonlinear model predictive control for dynamic decision problems in economics," Journal of Economic Dynamics and Control, Elsevier, vol. 60(C), pages 112-133.
    4. Duersch, Peter & Oechssler, Jörg & Schipper, Burkhard C., 2012. "Unbeatable imitation," Games and Economic Behavior, Elsevier, vol. 76(1), pages 88-96.
    5. Nowak, Andrzej S., 2008. "Equilibrium in a dynamic game of capital accumulation with the overtaking criterion," Economics Letters, Elsevier, vol. 99(2), pages 233-237, May.
    6. Lars Grüne & Christopher M. Kellett & Steven R. Weller, 2017. "On the Relation Between Turnpike Properties for Finite and Infinite Horizon Optimal Control Problems," Journal of Optimization Theory and Applications, Springer, vol. 173(3), pages 727-745, June.
    7. Arie Leizarowitz, 2003. "An Algorithm to Identify and Compute Average Optimal Policies in Multichain Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 28(3), pages 553-586, August.
    8. Arie Leizarowitz & Alexander J. Zaslavski, 2007. "Uniqueness and Stability of Optimal Policies of Finite State Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 32(1), pages 156-167, February.
    9. Duersch, Peter & Oechssler, Jörg & Schipper, Burkhard C., 2012. "Unbeatable imitation," Games and Economic Behavior, Elsevier, vol. 76(1), pages 88-96.
    10. Adam Jonsson, 2023. "An axiomatic approach to Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 97(1), pages 117-133, February.
    11. Vivek S. Borkar & Vladimir Gaitsgory, 2019. "Linear Programming Formulation of Long-Run Average Optimal Control Problem," Journal of Optimization Theory and Applications, Springer, vol. 181(1), pages 101-125, April.
    12. Alessandro Arlotto & J. Michael Steele, 2016. "A Central Limit Theorem for Temporally Nonhomogenous Markov Chains with Applications to Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1448-1468, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:dyngam:v:7:y:2017:i:2:d:10.1007_s13235-016-0186-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.