IDEAS home Printed from https://ideas.repec.org/p/bie/wpaper/475.html
   My bibliography  Save this paper

A limit theorem for Markov decision processes

Author

Listed:
  • Staudigl, Mathias

    (Center for Mathematical Economics, Bielefeld University)

Abstract

In this paper we prove a deterministic approximation theorem for a sequence of Markov decision processes with finitely many actions and general state spaces as they appear frequently in economics, game theory and operations research. Using viscosity solution methods no a-priori differentiabililty assumptions are imposed on the value function. Applications for this result can be found in large deviation theory, and some simple economic problems.

Suggested Citation

  • Staudigl, Mathias, 2014. "A limit theorem for Markov decision processes," Center for Mathematical Economics Working Papers 475, Center for Mathematical Economics, Bielefeld University.
    • Handle: RePEc:bie:wpaper:475
    as

    Download full text from publisher

    File URL: https://pub.uni-bielefeld.de/download/2674039/2901847
    File Function: First Version, 2013
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yuliy Sannikov & Andrzej Skrzypacz, 2010. "The Role of Information in Repeated Games With Frequent Actions," Econometrica, Econometric Society, vol. 78(3), pages 847-882, May.
    2. Johannes Hörner & Takuo Sugaya & Satoru Takahashi & Nicolas Vieille, 2011. "Recursive Methods in Discounted Stochastic Games: An Algorithm for δ→ 1 and a Folk Theorem," Econometrica, Econometric Society, vol. 79(4), pages 1277-1318, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aislinn Bohren, 2016. "Using Persistence to Generate Incentives in a Dynamic Moral Hazard Problem," PIER Working Paper Archive 16-024, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 15 Oct 2016.
    2. Fudenberg, Drew & Ishii, Yuhta & Kominers, Scott Duke, 2014. "Delayed-response strategies in repeated games with observation lags," Journal of Economic Theory, Elsevier, vol. 150(C), pages 487-514.
    3. Hörner, Johannes & Takahashi, Satoru, 2016. "How fast do equilibrium payoff sets converge in repeated games?," Journal of Economic Theory, Elsevier, vol. 165(C), pages 332-359.
    4. Daehyun Kim & Ichiro Obara, 2023. "Asymptotic Value of Monitoring Structures in Stochastic Games," Papers 2308.09211, arXiv.org, revised May 2025.
    5. Kranz, Sebastian, 2013. "Relational Contracting, Repeated Negotiations, and Hold-Up," VfS Annual Conference 2013 (Duesseldorf): Competition Policy and Regulation in a Global Economic Order 80047, Verein für Socialpolitik / German Economic Association.
    6. Susanne Goldlücke & Sebastian Kranz, 2018. "Discounted stochastic games with voluntary transfers," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(1), pages 235-263, July.
    7. Du, Chuang, 2012. "Solving payoff sets of perfect public equilibria: an example," MPRA Paper 38622, University Library of Munich, Germany.
    8. Drew Fudenberg & David Levine, 2008. "Corrigendum to "Continuous time limits of repeated games with imperfect public monitoring"," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 11(1), pages 237-237, January.
    9. Hartman-Glaser, Barney & Piskorski, Tomasz & Tchistyi, Alexei, 2012. "Optimal securitization with moral hazard," Journal of Financial Economics, Elsevier, vol. 104(1), pages 186-202.
    10. Fedor Iskhakov & John Rust & Bertel Schjerning, 2018. "The Dynamics Of Bertrand Price Competition With Cost‐Reducing Investments," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 59(4), pages 1681-1731, November.
    11. Kimmo Berg, 2016. "Elementary Subpaths in Discounted Stochastic Games," Dynamic Games and Applications, Springer, vol. 6(3), pages 304-323, September.
    12. Johannes Hörner & Satoru Takahashi & Nicolas Vieille, 2015. "Truthful Equilibria in Dynamic Bayesian Games," Econometrica, Econometric Society, vol. 83(5), pages 1795-1848, September.
    13. Dylan Possamai & Chiara Rossato, 2023. "Golden parachutes under the threat of accidents," Papers 2312.02101, arXiv.org, revised Dec 2024.
    14. Wälde, Klaus & Bayer, Christian, 2011. "Describing the Dynamics of Distribution in Search and Matching Models by Fokker-Planck Equations," VfS Annual Conference 2011 (Frankfurt, Main): The Order of the World Economy - Lessons from the Crisis 48736, Verein für Socialpolitik / German Economic Association.
    15. Henri Pages & Dylan Possamaï, 2014. "A mathematical treatment of bank monitoring incentives," Finance and Stochastics, Springer, vol. 18(1), pages 39-73, January.
    16. Barlo, Mehmet & Urgun, Can, 2011. "Stochastic discounting in repeated games: Awaiting the almost inevitable," MPRA Paper 28537, University Library of Munich, Germany.
    17. Johannes Horner & Satoru Takahashi & Nicolas Vieille, 2012. "On the Limit Equilibrium Payoff Set in Repeated and Stochastic Games," Cowles Foundation Discussion Papers 1848, Cowles Foundation for Research in Economics, Yale University.
    18. Osório Costa, Antonio Miguel, 2012. "The Limits of Discrete Time Repeated Games:Some Notes and Comments," Working Papers 2072/203171, Universitat Rovira i Virgili, Department of Economics.
    19. Aiba, Katsuhiko, 2014. "A folk theorem for stochastic games with private almost-perfect monitoring," Games and Economic Behavior, Elsevier, vol. 86(C), pages 58-66.
    20. Osório António M., 2012. "A Folk Theorem for Games when Frequent Monitoring Decreases Noise," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 12(1), pages 1-27, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bie:wpaper:475. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bettina Weingarten (email available below). General contact details of provider: https://edirc.repec.org/data/imbiede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.