IDEAS home Printed from https://ideas.repec.org/a/spr/jogath/v52y2023i4d10.1007_s00182-023-00860-5.html
   My bibliography  Save this article

Submixing and shift-invariant stochastic games

Author

Listed:
  • Hugo Gimbert

    (CNRS, LaBRI, Université de Bordeaux)

  • Edon Kelmendi

    (Queen Mary University of London)

Abstract

We study optimal strategies in two-player stochastic games that are played on a finite graph, equipped with a general payoff function. The existence of optimal strategies that do not make use of memory and randomisation is a desirable property that vastly simplifies the algorithmic analysis of such games. Our main theorem gives a sufficient condition for the maximizer to possess such a simple optimal strategy. The condition is imposed on the payoff function, saying the payoff does not depend on any finite prefix (shift-invariant) and combining two trajectories does not give higher payoff than the payoff of the parts (submixing). The core technical property that enables the proof of the main theorem is that of the existence of $$\epsilon$$ ϵ -subgame-perfect strategies when the payoff function is shift-invariant. Furthermore, the same techniques can be used to prove a finite-memory transfer-type theorem: namely that for shift-invariant and submixing payoff functions, the existence of optimal finite-memory strategies in one-player games for the minimizer implies the existence of the same in two-player games. We show that numerous classical payoff functions are submixing and shift-invariant.

Suggested Citation

  • Hugo Gimbert & Edon Kelmendi, 2023. "Submixing and shift-invariant stochastic games," International Journal of Game Theory, Springer;Game Theory Society, vol. 52(4), pages 1179-1214, December.
    • Handle: RePEc:spr:jogath:v:52:y:2023:i:4:d:10.1007_s00182-023-00860-5
    DOI: 10.1007/s00182-023-00860-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00182-023-00860-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00182-023-00860-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Cyrus Derman, 1962. "On Sequential Decisions and Markov Chains," Management Science, INFORMS, vol. 9(1), pages 16-24, October.
    2. Ayala Mashiah-Yaakovi, 2015. "Correlated Equilibria in Stochastic Games with Borel Measurable Payoffs," Dynamic Games and Applications, Springer, vol. 5(1), pages 120-135, March.
    3. Vrieze, O.J. & Tijs, S.H. & Raghavan, T.E.S. & Filar, J.A., 1983. "A finite algorithm for the switching control stochastic game," Other publications TiSEM 61df4c61-65ea-4357-99c0-1, Tilburg University, School of Economics and Management.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Oguzhan Alagoz & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2007. "Determining the Acceptance of Cadaveric Livers Using an Implicit Model of the Waiting List," Operations Research, INFORMS, vol. 55(1), pages 24-36, February.
    2. Lodewijk Kallenberg, 2013. "Derman’s book as inspiration: some results on LP for MDPs," Annals of Operations Research, Springer, vol. 208(1), pages 63-94, September.
    3. János Flesch & P. Jean-Jacques Herings & Jasmine Maes & Arkadi Predtetchinski, 2021. "Subgame Maxmin Strategies in Zero-Sum Stochastic Games with Tolerance Levels," Dynamic Games and Applications, Springer, vol. 11(4), pages 704-737, December.
    4. David L. Kaufman & Mark E. Lewis, 2007. "Machine maintenance with workload considerations," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(7), pages 750-766, October.
    5. Prasenjit Mondal, 2020. "Computing semi-stationary optimal policies for multichain semi-Markov decision processes," Annals of Operations Research, Springer, vol. 287(2), pages 843-865, April.
    6. Alexander Zadorojniy & Guy Even & Adam Shwartz, 2009. "A Strongly Polynomial Algorithm for Controlled Queues," Mathematics of Operations Research, INFORMS, vol. 34(4), pages 992-1007, November.
    7. K. Helmes & R. H. Stockbridge, 2000. "Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems," Journal of Optimization Theory and Applications, Springer, vol. 106(1), pages 107-127, July.
    8. Michael O’Sullivan & Arthur F. Veinott, Jr., 2017. "Polynomial-Time Computation of Strong and n -Present-Value Optimal Policies in Markov Decision Chains," Mathematics of Operations Research, INFORMS, vol. 42(3), pages 577-598, August.
    9. David T. Abdul‐Malak & Jeffrey P. Kharoufeh & Lisa M. Maillart, 2019. "Maintaining systems with heterogeneous spare parts," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(6), pages 485-501, September.
    10. Durango-Cohen, Pablo L., 2007. "A time series analysis framework for transportation infrastructure management," Transportation Research Part B: Methodological, Elsevier, vol. 41(5), pages 493-505, June.
    11. Guy Even & Alexander Zadorojniy, 2012. "Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks," Annals of Operations Research, Springer, vol. 201(1), pages 159-167, December.
    12. Michael Katehakis & Ingram Olkin & Sheldon Ross & Jian Yang, 2013. "On the life and work of Cyrus Derman," Annals of Operations Research, Springer, vol. 208(1), pages 5-26, September.
    13. Gomez-Ramirez, E. & Najim, K. & Poznyak, A. S., 2003. "Saddle-point calculation for constrained finite Markov chains," Journal of Economic Dynamics and Control, Elsevier, vol. 27(10), pages 1833-1853, August.
    14. Eugene A. Feinberg & Pavlo O. Kasyanov & Nina V. Zadoianchuk, 2012. "Average Cost Markov Decision Processes with Weakly Continuous Transition Probabilities," Mathematics of Operations Research, INFORMS, vol. 37(4), pages 591-607, November.
    15. Oguzhan Alagoz & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2007. "Choosing Among Living-Donor and Cadaveric Livers," Management Science, INFORMS, vol. 53(11), pages 1702-1715, November.
    16. Ahmadi, Reza & Newby, Martin, 2011. "Maintenance scheduling of a manufacturing system subject to deterioration," Reliability Engineering and System Safety, Elsevier, vol. 96(10), pages 1411-1420.
    17. Duvocelle, Benoit & Flesch, János & Staudigl, Mathias & Vermeulen, Dries, 2022. "A competitive search game with a moving target," European Journal of Operational Research, Elsevier, vol. 303(2), pages 945-957.
    18. Matthew Sobel, 2013. "Discounting axioms imply risk neutrality," Annals of Operations Research, Springer, vol. 208(1), pages 417-432, September.
    19. Shanshan Guo & Lei Zhao & Xiaowei Xu, 2016. "Impact of supply risks on procurement decisions," Annals of Operations Research, Springer, vol. 241(1), pages 411-430, June.
    20. B. Curtis Eaves & Arthur F. Veinott, 2014. "Maximum-Stopping-Value Policies in Finite Markov Population Decision Chains," Mathematics of Operations Research, INFORMS, vol. 39(3), pages 597-606, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jogath:v:52:y:2023:i:4:d:10.1007_s00182-023-00860-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.