IDEAS home Printed from
   My bibliography  Save this article

State of the Art---A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms


  • George E. Monahan

    (Georgia Institute of Technology)


This paper surveys models and algorithms dealing with partially observable Markov decision processes. A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process which permits uncertainty regarding the state of a Markov process and allows for state information acquisition. A general framework for finite state and action POMDP's is presented. Next, there is a brief discussion of the development of POMDP's and their relationship with other decision processes. A wide range of models in such areas as quality control, machine maintenance, internal auditing, learning, and optimal stopping are discussed within the POMDP-framework. Lastly, algorithms for computing optimal solutions to POMDP's are presented.

Suggested Citation

  • George E. Monahan, 1982. "State of the Art---A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms," Management Science, INFORMS, vol. 28(1), pages 1-16, January.
  • Handle: RePEc:inm:ormnsc:v:28:y:1982:i:1:p:1-16

    Download full text from publisher

    File URL:
    Download Restriction: no


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Ricardo Montoya & Oded Netzer & Kamel Jedidi, 2010. "Dynamic Allocation of Pharmaceutical Detailing and Sampling for Long-Term Profitability," Marketing Science, INFORMS, vol. 29(5), pages 909-924, 09-10.
    2. James T. Treharne & Charles R. Sox, 2002. "Adaptive Inventory Control for Nonstationary Demand and Partial Information," Management Science, INFORMS, vol. 48(5), pages 607-624, May.
    3. Gong, Linguo & Tang, Kwei, 1997. "Monitoring machine operations using on-line sensors," European Journal of Operational Research, Elsevier, vol. 96(3), pages 479-492, February.
    4. Givon, Moshe & Grosfeld-Nir, Abraham, 2008. "Using partially observed Markov processes to select optimal termination time of TV shows," Omega, Elsevier, vol. 36(3), pages 477-485, June.
    5. Lian, Zhaotong & Deshmukh, Abhijit, 2006. "Performance prediction of an unmanned airborne vehicle multi-agent system," European Journal of Operational Research, Elsevier, vol. 172(2), pages 680-695, July.
    6. Serin, Yasemin, 1995. "A nonlinear programming model for partially observable Markov decision processes: Finite horizon case," European Journal of Operational Research, Elsevier, vol. 86(3), pages 549-564, November.
    7. Jingyu Zhang & Brian T. Denton & Hari Balasubramanian & Nilay D. Shah & Brant A. Inman, 2012. "Optimization of Prostate Biopsy Referral Decisions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 529-547, October.
    8. Arifoglu, Kenan & Özekici, Süleyman, 2011. "Inventory management with random supply and imperfect information: A hidden Markov model," International Journal of Production Economics, Elsevier, vol. 134(1), pages 123-137, November.
    9. Kobayashi, Teruyoshi, 2009. "Announcements and the effectiveness of monetary policy: A view from the US prime rate," Journal of Banking & Finance, Elsevier, vol. 33(12), pages 2253-2266, December.
    10. repec:eee:reensy:v:130:y:2014:i:c:p:202-213 is not listed on IDEAS
    11. repec:eee:ecomod:v:220:y:2009:i:6:p:830-840 is not listed on IDEAS
    12. Chernonog, Tatyana & Avinadav, Tal, 2016. "A two-state partially observable Markov decision process with three actionsAuthor-Name: Ben-Zvi, Tal," European Journal of Operational Research, Elsevier, vol. 254(3), pages 957-967.
    13. repec:eee:ecomod:v:222:y:2011:i:5:p:1092-1102 is not listed on IDEAS
    14. repec:eee:reensy:v:167:y:2017:i:c:p:652-662 is not listed on IDEAS
    15. Baggio, Michele & Fackler, Paul L., 2016. "Optimal management with reversible regime shifts," Journal of Economic Behavior & Organization, Elsevier, vol. 132(PB), pages 124-136.
    16. repec:pal:jorsoc:v:57:y:2006:i:8:d:10.1057_palgrave.jors.2602048 is not listed on IDEAS
    17. Yossi Aviv & Amit Pazgal, 2005. "A Partially Observed Markov Decision Process for Dynamic Pricing," Management Science, INFORMS, vol. 51(9), pages 1400-1416, September.
    18. Arifoglu, Kenan & Özekici, Süleyman, 2010. "Optimal policies for inventory systems with finite capacity and partially observed Markov-modulated demand and supply processes," European Journal of Operational Research, Elsevier, vol. 204(3), pages 421-438, August.
    19. Fackler, Paul L. & Haight, Robert G., 2014. "Monitoring as a partially observable decision problem," Resource and Energy Economics, Elsevier, vol. 37(C), pages 226-241.
    20. White, Benedict, 2002. "Optimal Monitoring of Agri-environmental Schemes," 2002 Conference (46th), February 13-15, 2002, Canberra 125606, Australian Agricultural and Resource Economics Society.
    21. repec:pal:jorsoc:v:61:y:2010:i:2:d:10.1057_jors.2008.137 is not listed on IDEAS
    22. Haight, Robert G. & Polasky, Stephen, 2010. "Optimal control of an invasive species with imperfect information about the level of infestation," Resource and Energy Economics, Elsevier, vol. 32(4), pages 519-533, November.
    23. Grosfeld-Nir, Abraham, 2007. "Control limits for two-state partially observable Markov decision processes," European Journal of Operational Research, Elsevier, vol. 182(1), pages 300-304, October.
    24. Jang, Wooseung & Shanthikumar, J. George, 2004. "Sequential process control under capacity constraints," European Journal of Operational Research, Elsevier, vol. 155(3), pages 695-714, June.


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:28:y:1982:i:1:p:1-16. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mirko Janc). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.