IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v35y1987i5p736-743.html
   My bibliography  Save this article

Some Monotonicity Results for Partially Observed Markov Decision Processes

Author

Listed:
  • William S. Lovejoy

    (Stanford University, Stanford, California)

Abstract

This paper provides sufficient conditions for the optimal value in a discrete-time, finite, partially observed Markov decision process to be monotone on the space of state probability vectors ordered by likelihood ratios. The paper also presents sufficient conditions for the optimal policy to be monotone in a simple machine replacement problem, and, in the general case, for the optimal policy to be bounded from below by an easily calculated monotone function.

Suggested Citation

  • William S. Lovejoy, 1987. "Some Monotonicity Results for Partially Observed Markov Decision Processes," Operations Research, INFORMS, vol. 35(5), pages 736-743, October.
    • Handle: RePEc:inm:oropre:v:35:y:1987:i:5:p:736-743
    DOI: 10.1287/opre.35.5.736
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.35.5.736
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.35.5.736?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James T. Treharne & Charles R. Sox, 2002. "Adaptive Inventory Control for Nonstationary Demand and Partial Information," Management Science, INFORMS, vol. 48(5), pages 607-624, May.
    2. Deep, Akash & Zhou, Shiyu & Veeramani, Dharmaraj & Chen, Yong, 2023. "Partially observable Markov decision process-based optimal maintenance planning with time-dependent observations," European Journal of Operational Research, Elsevier, vol. 311(2), pages 533-544.
    3. Burhaneddin Sandıkçı & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2013. "Alleviating the Patient's Price of Privacy Through a Partially Observable Waiting List," Management Science, INFORMS, vol. 59(8), pages 1836-1854, August.
    4. Lisa M. Maillart & Ludmila Zheltova, 2007. "Structured maintenance policies on interior sample paths," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(6), pages 645-655, September.
    5. Li, Weiyu & Denton, Brian T. & Morgan, Todd M., 2023. "Optimizing active surveillance for prostate cancer using partially observable Markov decision processes," European Journal of Operational Research, Elsevier, vol. 305(1), pages 386-399.
    6. James E. Smith & Canan Ulu, 2017. "Risk Aversion, Information Acquisition, and Technology Adoption," Operations Research, INFORMS, vol. 65(4), pages 1011-1028, August.
    7. Wooseung Jang & J. George Shanthikumar, 2002. "Stochastic allocation of inspection capacity to competitive processes," Naval Research Logistics (NRL), John Wiley & Sons, vol. 49(1), pages 78-94, February.
    8. Hao Zhang, 2010. "Partially Observable Markov Decision Processes: A Geometric Technique and Analysis," Operations Research, INFORMS, vol. 58(1), pages 214-228, February.
    9. Jingyu Zhang & Brian T. Denton & Hari Balasubramanian & Nilay D. Shah & Brant A. Inman, 2012. "Optimization of Prostate Biopsy Referral Decisions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 529-547, October.
    10. L M Maillart & T G Yeung & Z Gozde Icten, 2011. "Selecting test sensitivity and specificity parameters to optimally maintain a degrading system," Journal of Risk and Reliability, , vol. 225(2), pages 131-139, June.
    11. Oussama Habachi & Yezekael Hayel & Rachid El-Azouzi, 2018. "Optimal energy-delay tradeoff for opportunistic spectrum access in cognitive radio networks," Telecommunication Systems: Modelling, Analysis, Design and Management, Springer, vol. 67(4), pages 763-780, April.
    12. Vikram Krishnamurthy & Sujay Bhatt, 2015. "Sequential Detection of Market shocks using Risk-averse Agent Based Models," Papers 1511.01965, arXiv.org.
    13. Lu Jin & Undarmaa Bayarsaikhan & Kazuyuki Suzuki, 2016. "Optimal control limit policy for age-dependent deteriorating systems under incomplete observations," Journal of Risk and Reliability, , vol. 230(1), pages 34-43, February.
    14. Saghafian, Soroush, 2018. "Ambiguous partially observable Markov decision processes: Structural results and applications," Journal of Economic Theory, Elsevier, vol. 178(C), pages 1-35.
    15. Sarang Deo & Seyed Iravani & Tingting Jiang & Karen Smilowitz & Stephen Samuelson, 2013. "Improving Health Outcomes Through Better Capacity Allocation in a Community-Based Chronic Care Model," Operations Research, INFORMS, vol. 61(6), pages 1277-1294, December.
    16. Miehling, Erik & Teneketzis, Demosthenis, 2020. "Monotonicity properties for two-action partially observable Markov decision processes on partially ordered spaces," European Journal of Operational Research, Elsevier, vol. 282(3), pages 936-944.
    17. Junbo Son & Yeongin Kim & Shiyu Zhou, 2022. "Alerting patients via health information system considering trust-dependent patient adherence," Information Technology and Management, Springer, vol. 23(4), pages 245-269, December.
    18. Sahin, Izzet & Zahedi, Fatemeh (Mariam), 2000. "Optimal policies under risk for changing software systems based on customer satisfaction," European Journal of Operational Research, Elsevier, vol. 123(1), pages 175-194, May.
    19. Chiel van Oosterom & Lisa M. Maillart & Jeffrey P. Kharoufeh, 2017. "Optimal maintenance policies for a safety‐critical system and its deteriorating sensor," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(5), pages 399-417, August.
    20. van Staden, Heletjé E. & Boute, Robert N., 2021. "The effect of multi-sensor data on condition-based maintenance policies," European Journal of Operational Research, Elsevier, vol. 290(2), pages 585-600.
    21. Hao Zhang & Weihua Zhang, 2023. "Analytical Solution to a Partially Observable Machine Maintenance Problem with Obvious Failures," Management Science, INFORMS, vol. 69(7), pages 3993-4015, July.
    22. Jue Wang & Chi-Guhn Lee, 2015. "Multistate Bayesian Control Chart Over a Finite Horizon," Operations Research, INFORMS, vol. 63(4), pages 949-964, August.
    23. Vikram Krishnamurthy & Udit Pareek, 2015. "Myopic Bounds for Optimal Policy of POMDPs: An Extension of Lovejoy’s Structural Results," Operations Research, INFORMS, vol. 63(2), pages 428-434, April.
    24. M. Reza Skandari & Steven M. Shechter, 2021. "Patient-Type Bayes-Adaptive Treatment Plans," Operations Research, INFORMS, vol. 69(2), pages 574-598, March.
    25. Vikram Krishnamurthy & Bo Wahlberg, 2009. "Partially Observed Markov Decision Process Multiarmed Bandits---Structural Results," Mathematics of Operations Research, INFORMS, vol. 34(2), pages 287-302, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:35:y:1987:i:5:p:736-743. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.