IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v222y2014i1p227-23710.1007-s10479-012-1302-3.html
   My bibliography  Save this article

Optimal halting policies in Markov population decision chains with constant risk posture

Author

Listed:
  • Pelin Canbolat

Abstract

This paper concerns infinite-horizon Markov population decision chains with finite state-action space, where one exerts control on a population of individuals in different states by assigning an action to each individual in the system in each period. In every transition, each individual earns a random reward and generates a random progeny vector. The objective of the decision maker is to maximize expected (infinite-horizon) system utility under the following assumptions: (i) The utility function exhibits constant risk posture, (ii) the (random) progeny vectors of distinct individuals are independent, and (iii) the progeny vectors of individuals in a state who take the same action are identically distributed. The paper deals with the problem of finding an optimal stationary halting policy and shows that this problem can be solved efficiently using successive approximations with the original state-action space without enlarging it to include information about the population in each state or any other aspect of the system history in a state. The proposed algorithm terminates after a finite number of iterations with an optimal stationary halting policy or with proof of nonexistence. Copyright Springer Science+Business Media New York 2014

Suggested Citation

  • Pelin Canbolat, 2014. "Optimal halting policies in Markov population decision chains with constant risk posture," Annals of Operations Research, Springer, vol. 222(1), pages 227-237, November.
    • Handle: RePEc:spr:annopr:v:222:y:2014:i:1:p:227-237:10.1007/s10479-012-1302-3
    DOI: 10.1007/s10479-012-1302-3
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-012-1302-3
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-012-1302-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Eric V. Denardo & Haechurl Park & Uriel G. Rothblum, 2007. "Risk-Sensitive and Risk-Neutral Multiarmed Bandits," Mathematics of Operations Research, INFORMS, vol. 32(2), pages 374-394, May.
    2. Ranel E. Erickson, 1988. "Optimality of Stationary Halting Policies and Finite Termination of Successive Approximations," Mathematics of Operations Research, INFORMS, vol. 13(1), pages 90-98, February.
    3. Rothblum, Uriel G., 1975. "Multivariate constant risk posture," Journal of Economic Theory, Elsevier, vol. 10(3), pages 309-332, June.
    4. Uriel G. Rothblum, 1984. "Multiplicative Markov Decision Chains," Mathematics of Operations Research, INFORMS, vol. 9(1), pages 6-24, February.
    5. Ethem Çanakoğlu & Süleyman Özekici, 2009. "Portfolio selection in stochastic markets with exponential utility functions," Annals of Operations Research, Springer, vol. 166(1), pages 281-297, February.
    6. Ronald A. Howard, 1971. "Proximal Decision Analysis," Management Science, INFORMS, vol. 17(9), pages 507-541, May.
    7. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    8. Giri, B.C., 2011. "Managing inventory with two suppliers under yield uncertainty and risk aversion," International Journal of Production Economics, Elsevier, vol. 133(1), pages 80-85, September.
    9. Choi, Sungyong & Ruszczynski, Andrzej, 2011. "A multi-product risk-averse newsvendor with exponential utility function," European Journal of Operational Research, Elsevier, vol. 214(1), pages 78-84, October.
    10. Eric V. Denardo & Uriel G. Rothblum & Ludo Van der Heyden, 2004. "Index Policies for Stochastic Search in a Forest with an Application to R&D Project Management," Mathematics of Operations Research, INFORMS, vol. 29(1), pages 162-181, February.
    11. Rolando Cavazos-Cadena & Raúl Montes-de-Oca, 2003. "The Value Iteration Algorithm in Risk-Sensitive Average Markov Decision Chains with Finite State Space," Mathematics of Operations Research, INFORMS, vol. 28(4), pages 752-776, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Amanda M. White & Pelin G. Canbolat, 2018. "Finite‐horizon Markov population decision chains with constant risk posture," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(8), pages 580-593, December.
    2. Rolando Cavazos-Cadena, 2018. "Characterization of the Optimal Risk-Sensitive Average Cost in Denumerable Markov Decision Chains," Mathematics of Operations Research, INFORMS, vol. 43(3), pages 1025-1050, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pelin G. Canbolat & Uriel G. Rothblum, 2019. "Constant risk aversion in stochastic contests with exponential completion times," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(1), pages 4-14, February.
    2. Kumar, Uday M & Bhat, Sanjay P. & Kavitha, Veeraruna & Hemachandra, Nandyala, 2023. "Approximate solutions to constrained risk-sensitive Markov decision processes," European Journal of Operational Research, Elsevier, vol. 310(1), pages 249-267.
    3. Basu, Arnab & Ghosh, Mrinal Kanti, 2014. "Zero-sum risk-sensitive stochastic games on a countable state space," Stochastic Processes and their Applications, Elsevier, vol. 124(1), pages 961-983.
    4. Chan, Chi Kin & Lee, Y.C.E. & Campbell, J.F., 2013. "Environmental performance—Impacts of vendor–buyer coordination," International Journal of Production Economics, Elsevier, vol. 145(2), pages 683-695.
    5. V. S. Borkar & S. P. Meyn, 2002. "Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost," Mathematics of Operations Research, INFORMS, vol. 27(1), pages 192-209, February.
    6. Chun-Hung Chiu & Tsan-Ming Choi, 2016. "Supply chain risk analysis with mean-variance models: a technical review," Annals of Operations Research, Springer, vol. 240(2), pages 489-507, May.
    7. Rolando Cavazos-Cadena, 2009. "Solutions of the average cost optimality equation for finite Markov decision chains: risk-sensitive and risk-neutral criteria," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 70(3), pages 541-566, December.
    8. Karel Sladký, 2013. "Risk-Sensitive and Mean Variance Optimality in Markov Decision Processes," Czech Economic Review, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, vol. 7(3), pages 146-161, November.
    9. Pestien, Victor & Wang, Xiaobo, 1998. "Markov-achievable payoffs for finite-horizon decision models," Stochastic Processes and their Applications, Elsevier, vol. 73(1), pages 101-118, January.
    10. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.
    11. HuiChen Chiang, 2007. "Financial intermediary's choice of borrowing," Applied Economics, Taylor & Francis Journals, vol. 40(2), pages 251-260.
    12. Hui Chen Chiang, 2007. "Optimal prepayment behaviour," Applied Economics Letters, Taylor & Francis Journals, vol. 14(15), pages 1127-1129.
    13. Takayuki Osogami, 2012. "Iterated risk measures for risk-sensitive Markov decision processes with discounted cost," Papers 1202.3755, arXiv.org.
    14. Amanda M. White & Pelin G. Canbolat, 2018. "Finite‐horizon Markov population decision chains with constant risk posture," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(8), pages 580-593, December.
    15. Arnab Basu & Mrinal K. Ghosh, 2018. "Nonzero-Sum Risk-Sensitive Stochastic Games on a Countable State Space," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 516-532, May.
    16. Zhang, Bin, 2012. "Multi-tier binary solution method for multi-product newsvendor problem with multiple constraints," European Journal of Operational Research, Elsevier, vol. 218(2), pages 426-434.
    17. Gokhan Metan & Aurélie Thiele, 2016. "Protecting the data-driven newsvendor against rare events: a correction-term approach," Computational Management Science, Springer, vol. 13(3), pages 459-482, July.
    18. Sonin, Isaac M., 2008. "A generalized Gittins index for a Markov chain and its recursive calculation," Statistics & Probability Letters, Elsevier, vol. 78(12), pages 1526-1533, September.
    19. Faiza Hamdi & Ahmed Ghorbel & Faouzi Masmoudi & Lionel Dupont, 2018. "Optimization of a supply portfolio in the context of supply chain risk management: literature review," Journal of Intelligent Manufacturing, Springer, vol. 29(4), pages 763-788, April.
    20. Jiarong Luo & Xu Chen & Chong Wang & Gaoxun Zhang, 2021. "Bidirectional options in random yield supply chains with demand and spot price uncertainty," Annals of Operations Research, Springer, vol. 302(1), pages 211-230, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:222:y:2014:i:1:p:227-237:10.1007/s10479-012-1302-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.