IDEAS home Printed from https://ideas.repec.org/a/spr/topjnl/v23y2015i3p799-836.html
   My bibliography  Save this article

Approximation of zero-sum continuous-time Markov games under the discounted payoff criterion

Author

Listed:
  • Tomás Prieto-Rumeau
  • José Lorenzo

Abstract

We deal with a two-person zero-sum continuous-time Markov game $$\mathcal {G}$$ G with denumerable state space, general action spaces, and unbounded payoff and transition rates. We consider noncooperative equilibria for the discounted payoff criterion. We are interested in approximating numerically the value and the optimal strategies of $$\mathcal {G}$$ G . To this end, we propose a definition of a sequence of game models $$\mathcal {G}_n$$ G n converging to $$\mathcal {G}$$ G , which ensures that the value and the optimal strategies of $$\mathcal {G}_n$$ G n converge to those of $$\mathcal {G}$$ G . For numerical purposes, we construct finite state and actions game models $$\mathcal {G}_n$$ G n that can be explicitly solved, and we study the convergence rate of the value of the games. A game model based on a population system illustrates our results. Copyright Sociedad de Estadística e Investigación Operativa 2015

Suggested Citation

  • Tomás Prieto-Rumeau & José Lorenzo, 2015. "Approximation of zero-sum continuous-time Markov games under the discounted payoff criterion," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(3), pages 799-836, October.
    • Handle: RePEc:spr:topjnl:v:23:y:2015:i:3:p:799-836
    DOI: 10.1007/s11750-014-0354-8
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11750-014-0354-8
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11750-014-0354-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. A. Jaśkiewicz & A. S. Nowak, 2006. "Approximation of Noncooperative Semi-Markov Games," Journal of Optimization Theory and Applications, Springer, vol. 131(1), pages 115-134, October.
    2. Guo, Xianping & Zhang, Wenzhao, 2014. "Convergence of controlled models and finite-state approximation for discounted continuous-time Markov decision processes with constraints," European Journal of Operational Research, Elsevier, vol. 238(2), pages 486-496.
    3. Frenk, J. B. G. & Kassay, G. & Kolumban, J., 2004. "On equivalent results in minimax theory," European Journal of Operational Research, Elsevier, vol. 157(1), pages 46-58, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. François Dufour & Tomás Prieto-Rumeau, 2019. "Approximation of Discounted Minimax Markov Control Problems and Zero-Sum Markov Games Using Hausdorff and Wasserstein Distances," Dynamic Games and Applications, Springer, vol. 9(1), pages 68-102, March.
    2. Yonghui Huang & Zhaotong Lian & Xianping Guo, 2023. "Zero-sum infinite-horizon discounted piecewise deterministic Markov games," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 97(2), pages 179-205, April.
    3. Xianping Guo & Yi Zhang, 2016. "Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1276-1296, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frenk, J.B.G. & Kassay, G., 2005. "Lagrangian duality and cone convexlike functions," ERIM Report Series Research in Management ERS-2005-019-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    2. Frenk, J.B.G. & Schaible, S., 2004. "Fractional Programming," ERIM Report Series Research in Management ERS-2004-074-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    3. Frenk, J.B.G. & Kassay, G., 2006. "On noncooperative games, minimax theorems and equilibrium problems," Econometric Institute Research Papers EI 2006-21, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    4. Mingchao Xia & Qingying Lai & Yajiao Zhong & Canbing Li & Hsiao-Dong Chiang, 2016. "Aggregator-Based Interactive Charging Management System for Electric Vehicle Charging," Energies, MDPI, vol. 9(3), pages 1-14, March.
    5. Fernando Luque-Vásquez & J. Adolfo Minjárez-Sosa & Max E. Mitre-Báez, 2016. "A Note on König and Close Convexity in Minimax Theorems," Journal of Optimization Theory and Applications, Springer, vol. 170(1), pages 65-71, July.
    6. Frenk, J.B.G. & Schaible, S., 2004. "Fractional Programming," Econometric Institute Research Papers ERS-2004-074-LIS, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    7. Łukasz Balbus & Kevin Reffett & Łukasz Woźny, 2013. "Markov Stationary Equilibria in Stochastic Supermodular Games with Imperfect Private and Public Information," Dynamic Games and Applications, Springer, vol. 3(2), pages 187-206, June.
    8. H. Boualam & A. Roubi, 2019. "Proximal bundle methods based on approximate subgradients for solving Lagrangian duals of minimax fractional programs," Journal of Global Optimization, Springer, vol. 74(2), pages 255-284, June.
    9. Frenk, J.B.G. & Kassay, G., 2005. "On Noncooperative Games and Minimax Theory," ERIM Report Series Research in Management ERS-2005-036-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    10. Qingda Wei, 2017. "Finite approximation for finite-horizon continuous-time Markov decision processes," 4OR, Springer, vol. 15(1), pages 67-84, March.
    11. Ping Cao & Jingui Xie, 2016. "Optimal control of a multiclass queueing system when customers can change types," Queueing Systems: Theory and Applications, Springer, vol. 82(3), pages 285-313, April.
    12. Frenk, J.B.G. & Kassay, G., 2006. "On Noncooperative Games, Minimax Theorems and Equilibrium Problems," ERIM Report Series Research in Management ERS-2006-022-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    13. Qingda Wei, 2016. "Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 84(3), pages 461-487, December.
    14. David González-Sánchez & Fernando Luque-Vásquez & J. Adolfo Minjárez-Sosa, 2019. "Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies," Dynamic Games and Applications, Springer, vol. 9(1), pages 103-121, March.
    15. Dimitris Bertsimas & Ebrahim Nasrabadi & Sebastian Stiller, 2013. "Robust and Adaptive Network Flows," Operations Research, INFORMS, vol. 61(5), pages 1218-1242, October.
    16. J. B. G. Frenk & G. Kassay, 2007. "Lagrangian Duality and Cone Convexlike Functions," Journal of Optimization Theory and Applications, Springer, vol. 134(2), pages 207-222, August.
    17. Frenk, J.B.G. & Kassay, G., 2004. "Introduction to Convex and Quasiconvex Analysis," ERIM Report Series Research in Management ERS-2004-075-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    18. J. B. G. Frenk & P. Kas & G. Kassay, 2007. "On Linear Programming Duality and Necessary and Sufficient Conditions in Minimax Theory," Journal of Optimization Theory and Applications, Springer, vol. 132(3), pages 423-439, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:topjnl:v:23:y:2015:i:3:p:799-836. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.