IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v27y1979i5p1041-1053.html
   My bibliography  Save this article

Structural Results for Partially Observable Markov Decision Processes

Author

Listed:
  • S. Christian Albright

    (Indiana University, Bloomington, Indiana)

Abstract

This paper examines monotonicity results for a fairly general class of partially observable Markov decision processes. When there are only two actual states in the system and when the actions taken are primarily intended to improve the system, rather than to inspect it, we give reasonable conditions which ensure that the optimal reward function and the optimal action are both monotone in the current state of information. Examples of maintenance systems and advertising systems for which our results hold are given. Finally, we examine the case where there are three or more actual states and indicate the difficulties encountered when we attempt to extend the monotonicity results to this situation.

Suggested Citation

  • S. Christian Albright, 1979. "Structural Results for Partially Observable Markov Decision Processes," Operations Research, INFORMS, vol. 27(5), pages 1041-1053, October.
    • Handle: RePEc:inm:oropre:v:27:y:1979:i:5:p:1041-1053
    DOI: 10.1287/opre.27.5.1041
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.27.5.1041
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.27.5.1041?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Givon, Moshe & Grosfeld-Nir, Abraham, 2008. "Using partially observed Markov processes to select optimal termination time of TV shows," Omega, Elsevier, vol. 36(3), pages 477-485, June.
    2. Li, Weiyu & Denton, Brian T. & Morgan, Todd M., 2023. "Optimizing active surveillance for prostate cancer using partially observable Markov decision processes," European Journal of Operational Research, Elsevier, vol. 305(1), pages 386-399.
    3. Armando Z. Milioni & Stanley R. Pliska, 1988. "Optimal inspection under semi‐markovian deterioration: Basic results," Naval Research Logistics (NRL), John Wiley & Sons, vol. 35(5), pages 373-392, October.
    4. Jingyu Zhang & Brian T. Denton & Hari Balasubramanian & Nilay D. Shah & Brant A. Inman, 2012. "Optimization of Prostate Biopsy Referral Decisions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 529-547, October.
    5. Shoshana Anily & Abraham Grosfeld-Nir, 2006. "An Optimal Lot-Sizing and Offline Inspection Policy in the Case of Nonrigid Demand," Operations Research, INFORMS, vol. 54(2), pages 311-323, April.
    6. Chernonog, Tatyana & Avinadav, Tal, 2016. "A two-state partially observable Markov decision process with three actionsAuthor-Name: Ben-Zvi, Tal," European Journal of Operational Research, Elsevier, vol. 254(3), pages 957-967.
    7. Abraham Grosfeld‐Nir & Eyal Cohen & Yigal Gerchak, 2007. "Production to order and off‐line inspection when the production process is partially observable," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(8), pages 845-858, December.
    8. Miehling, Erik & Teneketzis, Demosthenis, 2020. "Monotonicity properties for two-action partially observable Markov decision processes on partially ordered spaces," European Journal of Operational Research, Elsevier, vol. 282(3), pages 936-944.
    9. Chiel van Oosterom & Lisa M. Maillart & Jeffrey P. Kharoufeh, 2017. "Optimal maintenance policies for a safety‐critical system and its deteriorating sensor," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(5), pages 399-417, August.
    10. Ciriaco Valdez‐Flores & Richard M. Feldman, 1989. "A survey of preventive maintenance models for stochastically deteriorating single‐unit systems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 36(4), pages 419-446, August.
    11. Grosfeld-Nir, Abraham, 2007. "Control limits for two-state partially observable Markov decision processes," European Journal of Operational Research, Elsevier, vol. 182(1), pages 300-304, October.
    12. K. Waldmann, 1982. "On two-state quality control under Markovian deterioration," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 29(1), pages 249-260, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:27:y:1979:i:5:p:1041-1053. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.