IDEAS home Printed from https://ideas.repec.org/a/spr/jogath/v52y2023i4d10.1007_s00182-023-00860-5.html
   My bibliography  Save this article

Submixing and shift-invariant stochastic games

Author

Listed:
  • Hugo Gimbert

    (CNRS, LaBRI, Université de Bordeaux)

  • Edon Kelmendi

    (Queen Mary University of London)

Abstract

We study optimal strategies in two-player stochastic games that are played on a finite graph, equipped with a general payoff function. The existence of optimal strategies that do not make use of memory and randomisation is a desirable property that vastly simplifies the algorithmic analysis of such games. Our main theorem gives a sufficient condition for the maximizer to possess such a simple optimal strategy. The condition is imposed on the payoff function, saying the payoff does not depend on any finite prefix (shift-invariant) and combining two trajectories does not give higher payoff than the payoff of the parts (submixing). The core technical property that enables the proof of the main theorem is that of the existence of $$\epsilon$$ ϵ -subgame-perfect strategies when the payoff function is shift-invariant. Furthermore, the same techniques can be used to prove a finite-memory transfer-type theorem: namely that for shift-invariant and submixing payoff functions, the existence of optimal finite-memory strategies in one-player games for the minimizer implies the existence of the same in two-player games. We show that numerous classical payoff functions are submixing and shift-invariant.

Suggested Citation

  • Hugo Gimbert & Edon Kelmendi, 2023. "Submixing and shift-invariant stochastic games," International Journal of Game Theory, Springer;Game Theory Society, vol. 52(4), pages 1179-1214, December.
    • Handle: RePEc:spr:jogath:v:52:y:2023:i:4:d:10.1007_s00182-023-00860-5
    DOI: 10.1007/s00182-023-00860-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00182-023-00860-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00182-023-00860-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ayala Mashiah-Yaakovi, 2015. "Correlated Equilibria in Stochastic Games with Borel Measurable Payoffs," Dynamic Games and Applications, Springer, vol. 5(1), pages 120-135, March.
    2. Cyrus Derman, 1962. "On Sequential Decisions and Markov Chains," Management Science, INFORMS, vol. 9(1), pages 16-24, October.
    3. Vrieze, O.J. & Tijs, S.H. & Raghavan, T.E.S. & Filar, J.A., 1983. "A finite algorithm for the switching control stochastic game," Other publications TiSEM 61df4c61-65ea-4357-99c0-1, Tilburg University, School of Economics and Management.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lodewijk Kallenberg, 2013. "Derman’s book as inspiration: some results on LP for MDPs," Annals of Operations Research, Springer, vol. 208(1), pages 63-94, September.
    2. Ahmadi, Reza & Newby, Martin, 2011. "Maintenance scheduling of a manufacturing system subject to deterioration," Reliability Engineering and System Safety, Elsevier, vol. 96(10), pages 1411-1420.
    3. Shanshan Guo & Lei Zhao & Xiaowei Xu, 2016. "Impact of supply risks on procurement decisions," Annals of Operations Research, Springer, vol. 241(1), pages 411-430, June.
    4. Prasenjit Mondal, 2016. "On undiscounted semi-Markov decision processes with absorbing states," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 83(2), pages 161-177, April.
    5. Reza Ahmadi, 2014. "Optimal maintenance scheduling for a complex manufacturing system subject to deterioration," Annals of Operations Research, Springer, vol. 217(1), pages 1-29, June.
    6. Hui Zhang & Christian Wernz & Danny R. Hughes, 2018. "A Stochastic Game Analysis of Incentives and Behavioral Barriers in Chronic Disease Management," Service Science, INFORMS, vol. 10(3), pages 302-319, September.
    7. Durango-Cohen, Pablo L. & Madanat, Samer M., 2008. "Optimization of inspection and maintenance decisions for infrastructure facilities under performance model uncertainty: A quasi-Bayes approach," Transportation Research Part A: Policy and Practice, Elsevier, vol. 42(8), pages 1074-1085, October.
    8. Panagiotidou, Sofia & Nenes, George, 2009. "An economically designed, integrated quality and maintenance model using an adaptive Shewhart chart," Reliability Engineering and System Safety, Elsevier, vol. 94(3), pages 732-741.
    9. Prasenjit Mondal, 2020. "Computing semi-stationary optimal policies for multichain semi-Markov decision processes," Annals of Operations Research, Springer, vol. 287(2), pages 843-865, April.
    10. K. Helmes & R. H. Stockbridge, 2000. "Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems," Journal of Optimization Theory and Applications, Springer, vol. 106(1), pages 107-127, July.
    11. Michael Katehakis & Ingram Olkin & Sheldon Ross & Jian Yang, 2013. "On the life and work of Cyrus Derman," Annals of Operations Research, Springer, vol. 208(1), pages 5-26, September.
    12. Eugene A. Feinberg & Pavlo O. Kasyanov & Nina V. Zadoianchuk, 2012. "Average Cost Markov Decision Processes with Weakly Continuous Transition Probabilities," Mathematics of Operations Research, INFORMS, vol. 37(4), pages 591-607, November.
    13. Matthew Sobel, 2013. "Discounting axioms imply risk neutrality," Annals of Operations Research, Springer, vol. 208(1), pages 417-432, September.
    14. Chutian Ma & Paul Smith, 2025. "On the Effect of Alpha Decay and Transaction Costs on the Multi-period Optimal Trading Strategy," Papers 2502.04284, arXiv.org.
    15. Florio, Alexandre M. & Hartl, Richard F. & Minner, Stefan, 2020. "Optimal a priori tour and restocking policy for the single-vehicle routing problem with stochastic demands," European Journal of Operational Research, Elsevier, vol. 285(1), pages 172-182.
    16. Panagiotidou, Sofia & Tagaras, George, 2007. "Optimal preventive maintenance for equipment with two quality states and general failure time distributions," European Journal of Operational Research, Elsevier, vol. 180(1), pages 329-353, July.
    17. Dipti Dubey & S. K. Neogy & Debasish Ghorui, 2017. "Completely Mixed Strategies for Generalized Bimatrix and Switching Controller Stochastic Game," Dynamic Games and Applications, Springer, vol. 7(4), pages 535-554, December.
    18. Alan P. Wood, 1988. "Optimal maintenance policies for constantly monitored systems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 35(4), pages 461-471, August.
    19. Warren B. Powell, 2016. "Perspectives of approximate dynamic programming," Annals of Operations Research, Springer, vol. 241(1), pages 319-356, June.
    20. Oguzhan Alagoz & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2007. "Determining the Acceptance of Cadaveric Livers Using an Implicit Model of the Waiting List," Operations Research, INFORMS, vol. 55(1), pages 24-36, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jogath:v:52:y:2023:i:4:d:10.1007_s00182-023-00860-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.