IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v142y2006i1p165-18610.1007-s10479-006-6167-x.html
   My bibliography  Save this article

Approximating infinite horizon stochastic optimal control in discrete time with constraints

Author

Listed:
  • Lisa Korf

Abstract

Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are retained, and numerical solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimization problems with discounted cost criterion. Constraints on both states and controls are permitted, and modeled in the objective function by allowing it to take infinite values. Approximating techniques are developed using variational analysis, and intuitive lower bounds are obtained via averaging the future. These bounds could be used in a finite-time horizon stochastic programming setting to find solutions numerically. Copyright Springer Science + Business Media, Inc. 2006

Suggested Citation

  • Lisa Korf, 2006. "Approximating infinite horizon stochastic optimal control in discrete time with constraints," Annals of Operations Research, Springer, vol. 142(1), pages 165-186, February.
    • Handle: RePEc:spr:annopr:v:142:y:2006:i:1:p:165-186:10.1007/s10479-006-6167-x
    DOI: 10.1007/s10479-006-6167-x
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-006-6167-x
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-006-6167-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ward Whitt, 1979. "Approximations of Dynamic Programs, II," Mathematics of Operations Research, INFORMS, vol. 4(2), pages 179-185, May.
    2. GRINOLD, Richard C., 1977. "Finite horizon approximations of infinite horizon linear programs," LIDAM Reprints CORE 294, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Daniel Kuhn, 2009. "An Information-Based Approximation Scheme for Stochastic Optimization Problems in Continuous Time," Mathematics of Operations Research, INFORMS, vol. 34(2), pages 428-444, May.
    2. Armando F. Mendoza-Pérez & Héctor Jasso-Fuentes & Omar A. De-la-Cruz Courtois, 2016. "Constrained Markov decision processes in Borel spaces: from discounted to average optimality," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 84(3), pages 489-525, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert Kirkby Author-Email: robertkirkby@gmail.com|, 2017. "Convergence of Discretized Value Function Iteration," Computational Economics, Springer;Society for Computational Economics, vol. 49(1), pages 117-153, January.
    2. L.A. Korf, 1998. "Insurer's Portfolios of Risks: Approximating Infinite Horizon Stochastic Dynamic Optimization Problems," Working Papers ir98061, International Institute for Applied Systems Analysis.
    3. Ghate, Archis, 2015. "Circumventing the Slater conundrum in countably infinite linear programs," European Journal of Operational Research, Elsevier, vol. 246(3), pages 708-720.
    4. So, Wai Ping & Wan, Yat-wah, 2000. "A multi-grid size dynamic programming approach for the production control of a random-speed machine," International Journal of Production Economics, Elsevier, vol. 63(3), pages 267-275, January.
    5. David R. Cariño & David H. Myers & William T. Ziemba, 1998. "Concepts, Technical Issues, and Uses of the Russell-Yasuda Kasai Financial Planning Model," Operations Research, INFORMS, vol. 46(4), pages 450-462, August.
    6. Jie Ning & Matthew J. Sobel, 2019. "Easy Affine Markov Decision Processes," Operations Research, INFORMS, vol. 67(6), pages 1719-1737, November.
    7. Nicola Secomandi & François Margot, 2009. "Reoptimization Approaches for the Vehicle-Routing Problem with Stochastic Demands," Operations Research, INFORMS, vol. 57(1), pages 214-230, February.
    8. Knolmayer, Gerhard, 1981. "A simulation study of simplification strategies in the development of optimization models," Manuskripte aus den Instituten für Betriebswirtschaftslehre der Universität Kiel 96, Christian-Albrechts-Universität zu Kiel, Institut für Betriebswirtschaftslehre.
    9. Runggaldier, Wolfgang J., 1998. "Concepts and methods for discrete and continuous time control under uncertainty," Insurance: Mathematics and Economics, Elsevier, vol. 22(1), pages 25-39, May.
    10. H. Edwin Romeijn & Robert L. Smith, 1998. "Shadow Prices in Infinite-Dimensional Linear Programming," Mathematics of Operations Research, INFORMS, vol. 23(1), pages 239-256, February.
    11. Anton J. Kleywegt & Vijay S. Nori & Martin W. P. Savelsbergh, 2004. "Dynamic Programming Approximations for a Stochastic Inventory Routing Problem," Transportation Science, INFORMS, vol. 38(1), pages 42-70, February.
    12. William B. Haskell & Rahul Jain & Dileep Kalathil, 2016. "Empirical Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 41(2), pages 402-429, May.
    13. Naci Saldi & Serdar Yüksel & Tamás Linder, 2017. "On the Asymptotic Optimality of Finite Approximations to Markov Decision Processes with Borel Spaces," Mathematics of Operations Research, INFORMS, vol. 42(4), pages 945-978, November.
    14. Archis Ghate & Dushyant Sharma & Robert L. Smith, 2010. "A Shadow Simplex Method for Infinite Linear Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 865-877, August.
    15. Archis Ghate & Robert L. Smith, 2013. "A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes," Operations Research, INFORMS, vol. 61(2), pages 413-425, April.
    16. Michael Z. Spivey & Warren B. Powell, 2004. "The Dynamic Assignment Problem," Transportation Science, INFORMS, vol. 38(4), pages 399-419, November.
    17. Thomas W.M. Vossen & Fan You & Dan Zhang, 2022. "Finite‐horizon approximate linear programs for capacity allocation over a rolling horizon," Production and Operations Management, Production and Operations Management Society, vol. 31(5), pages 2127-2142, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:142:y:2006:i:1:p:165-186:10.1007/s10479-006-6167-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.