IDEAS home Printed from https://ideas.repec.org/p/bie/wpaper/475.html
   My bibliography  Save this paper

A limit theorem for Markov decision processes

Author

Listed:
  • Staudigl, Mathias

    (Center for Mathematical Economics, Bielefeld University)

Abstract

In this paper we prove a deterministic approximation theorem for a sequence of Markov decision processes with finitely many actions and general state spaces as they appear frequently in economics, game theory and operations research. Using viscosity solution methods no a-priori differentiabililty assumptions are imposed on the value function. Applications for this result can be found in large deviation theory, and some simple economic problems.

Suggested Citation

  • Staudigl, Mathias, 2014. "A limit theorem for Markov decision processes," Center for Mathematical Economics Working Papers 475, Center for Mathematical Economics, Bielefeld University.
    • Handle: RePEc:bie:wpaper:475
    as

    Download full text from publisher

    File URL: https://pub.uni-bielefeld.de/download/2674039/2901847
    File Function: First Version, 2013
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yuliy Sannikov & Andrzej Skrzypacz, 2010. "The Role of Information in Repeated Games With Frequent Actions," Econometrica, Econometric Society, vol. 78(3), pages 847-882, May.
    2. Johannes Hörner & Takuo Sugaya & Satoru Takahashi & Nicolas Vieille, 2011. "Recursive Methods in Discounted Stochastic Games: An Algorithm for δ→ 1 and a Folk Theorem," Econometrica, Econometric Society, vol. 79(4), pages 1277-1318, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fudenberg, Drew & Ishii, Yuhta & Kominers, Scott Duke, 2014. "Delayed-response strategies in repeated games with observation lags," Journal of Economic Theory, Elsevier, vol. 150(C), pages 487-514.
    2. Hörner, Johannes & Takahashi, Satoru, 2016. "How fast do equilibrium payoff sets converge in repeated games?," Journal of Economic Theory, Elsevier, vol. 165(C), pages 332-359.
    3. Daehyun Kim & Ichiro Obara, 2023. "Asymptotic Value of Monitoring Structures in Stochastic Games," Papers 2308.09211, arXiv.org, revised Jul 2024.
    4. Aislinn Bohren, 2016. "Using Persistence to Generate Incentives in a Dynamic Moral Hazard Problem," PIER Working Paper Archive 16-024, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 15 Oct 2016.
    5. Kimmo Berg & Gijs Schoenmakers, 2017. "Construction of Subgame-Perfect Mixed-Strategy Equilibria in Repeated Games," Games, MDPI, vol. 8(4), pages 1-14, November.
    6. , & ,, 2015. "A folk theorem for stochastic games with infrequent state changes," Theoretical Economics, Econometric Society, vol. 10(1), January.
    7. Johannes Hörner & Nicolas Klein & Sven Rady, 2022. "Overcoming Free-Riding in Bandit Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(4), pages 1948-1992.
    8. Kranz, Sebastian, 2013. "Relational Contracting, Repeated Negotiations, and Hold-Up," VfS Annual Conference 2013 (Duesseldorf): Competition Policy and Regulation in a Global Economic Order 80047, Verein für Socialpolitik / German Economic Association.
    9. Drew Fudenberg & David K. Levine, 2008. "Continuous time limits of repeated games with imperfect public monitoring," World Scientific Book Chapters, in: Drew Fudenberg & David K Levine (ed.), A Long-Run Collaboration On Long-Run Games, chapter 17, pages 369-388, World Scientific Publishing Co. Pte. Ltd..
    10. Renault, Jérôme & Solan, Eilon & Vieille, Nicolas, 2013. "Dynamic sender–receiver games," Journal of Economic Theory, Elsevier, vol. 148(2), pages 502-534.
    11. Susanne Goldlücke & Sebastian Kranz, 2018. "Discounted stochastic games with voluntary transfers," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(1), pages 235-263, July.
    12. Eilon Solan, 2018. "The modified stochastic game," International Journal of Game Theory, Springer;Game Theory Society, vol. 47(4), pages 1287-1327, November.
    13. Fudenberg, Drew & Olszewski, Wojciech, 2011. "Repeated games with asynchronous monitoring of an imperfect signal," Games and Economic Behavior, Elsevier, vol. 72(1), pages 86-99, May.
    14. Du, Chuang, 2012. "Solving payoff sets of perfect public equilibria: an example," MPRA Paper 38622, University Library of Munich, Germany.
    15. Pierre Cardaliaguet & Catherine Rainer & Dinah Rosenberg & Nicolas Vieille, 2016. "Markov Games with Frequent Actions and Incomplete Information—The Limit Case," Mathematics of Operations Research, INFORMS, vol. 41(1), pages 49-71, February.
    16. Hartman-Glaser, Barney & Piskorski, Tomasz & Tchistyi, Alexei, 2012. "Optimal securitization with moral hazard," Journal of Financial Economics, Elsevier, vol. 104(1), pages 186-202.
    17. Dilip Abreu & Benjamin Brooks & Yuliy Sannikov, 2016. "A "Pencil Sharpening" Algorithm for Two Player Stochastic Games with Perfect Monitoring," Working Papers 78_2016, Princeton University, Department of Economics, Econometric Research Program..
    18. Fedor Iskhakov & John Rust & Bertel Schjerning, 2018. "The Dynamics Of Bertrand Price Competition With Cost‐Reducing Investments," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 59(4), pages 1681-1731, November.
    19. Kimmo Berg, 2016. "Elementary Subpaths in Discounted Stochastic Games," Dynamic Games and Applications, Springer, vol. 6(3), pages 304-323, September.
    20. Piskorski, Tomasz & Westerfield, Mark M., 2016. "Optimal dynamic contracts with moral hazard and costly monitoring," Journal of Economic Theory, Elsevier, vol. 166(C), pages 242-281.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bie:wpaper:475. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bettina Weingarten (email available below). General contact details of provider: https://edirc.repec.org/data/imbiede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.