IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v235y2015i1p129-15310.1007-s10479-015-1905-6.html
   My bibliography  Save this article

Value of information for a leader–follower partially observed Markov game

Author

Listed:
  • Yanling Chang
  • Alan Erera
  • Chelsea White

Abstract

We consider a leader–follower partially observed Markov game (POMG) and analyze how the value of the leader’s criterion changes due to changes in the leader’s quality of observation of the follower. We give conditions that insure improved observation quality will improve the leader’s value function, assuming that changes in the observation quality do not cause the follower to change its policy. We show that discontinuities in the value of the leader’s criterion, as a function of observation quality, can occur when the change of observation quality is significant enough for the follower to change its policy. We present conditions that determine when a discontinuity may occur and conditions that guarantee a discontinuity will not degrade the leader’s performance. We show that when the leader and the follower are collaborative and the follower completely observes the leader’s initial state, discontinuities in the leader’s value function will not occur. However, examples show that improving observation quality does not necessarily improve the leader’s criterion value, whether or not the POMG is a collaborative game. Copyright Springer Science+Business Media New York 2015

Suggested Citation

  • Yanling Chang & Alan Erera & Chelsea White, 2015. "Value of information for a leader–follower partially observed Markov game," Annals of Operations Research, Springer, vol. 235(1), pages 129-153, December.
    • Handle: RePEc:spr:annopr:v:235:y:2015:i:1:p:129-153:10.1007/s10479-015-1905-6
    DOI: 10.1007/s10479-015-1905-6
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-015-1905-6
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-015-1905-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mingming Leng & Mahmut Parlar, 2009. "Allocation of Cost Savings in a Three-Level Supply Chain with Demand Information Sharing: A Cooperative-Game Approach," Operations Research, INFORMS, vol. 57(1), pages 200-213, February.
    2. Dinah Rosenberg & Bernard de Meyer & Ehud Lehrer, 2010. "Evaluating Information in Zero-Sum Games with Incomplete Information on Both Sides," Post-Print hal-00586037, HAL.
    3. Lehrer, Ehud & Rosenberg, Dinah, 2006. "What restrictions do Bayesian games impose on the value of information?," Journal of Mathematical Economics, Elsevier, vol. 42(3), pages 343-357, June.
    4. Vicki Bier & Santiago Oliveros & Larry Samuelson, 2007. "Choosing What to Protect: Strategic Defensive Allocation against an Unknown Attacker," Journal of Public Economic Theory, Association for Public Economic Theory, vol. 9(4), pages 563-587, August.
    5. Barry Charles Ezell & Steven P. Bennett & Detlof Von Winterfeldt & John Sokolowski & Andrew J. Collins, 2010. "Probabilistic Risk Analysis and Terrorism Risk," Risk Analysis, John Wiley & Sons, vol. 30(4), pages 575-589, April.
    6. Hao Zhang, 2010. "Partially Observable Markov Decision Processes: A Geometric Technique and Analysis," Operations Research, INFORMS, vol. 58(1), pages 214-228, February.
    7. Bernard De Meyer & Ehud Lehrer & Dinah Rosenberg, 2009. "Evaluating information in zero-sum games with incomplete information on both sides," Documents de travail du Centre d'Economie de la Sorbonne 09035, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    8. White, Chelsea C. & White, Douglas J., 1989. "Markov decision processes," European Journal of Operational Research, Elsevier, vol. 39(1), pages 1-16, March.
    9. Richard D. Smallwood & Edward J. Sondik, 1973. "The Optimal Control of Partially Observable Markov Processes over a Finite Horizon," Operations Research, INFORMS, vol. 21(5), pages 1071-1088, October.
    10. Lode Li, 2002. "Information Sharing in a Supply Chain with Horizontal Competition," Management Science, INFORMS, vol. 48(9), pages 1196-1212, September.
    11. Chelsea C. White & William T. Scherer, 1989. "Solution Procedures for Partially Observed Markov Decision Processes," Operations Research, INFORMS, vol. 37(5), pages 791-797, October.
    12. Zhuang, Jun & Bier, Vicki M. & Alagoz, Oguzhan, 2010. "Modeling secrecy and deception in a multiple-period attacker-defender signaling game," European Journal of Operational Research, Elsevier, vol. 203(2), pages 409-418, June.
    13. Daniel S. Bernstein & Robert Givan & Neil Immerman & Shlomo Zilberstein, 2002. "The Complexity of Decentralized Control of Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 27(4), pages 819-840, November.
    14. Bruno Bassan & Olivier Gossner & Marco Scarsini & Shmuel Zamir, 2003. "Positive value of information in games," International Journal of Game Theory, Springer;Game Theory Society, vol. 32(1), pages 17-31, December.
    15. George E. Monahan, 1982. "State of the Art---A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms," Management Science, INFORMS, vol. 28(1), pages 1-16, January.
    16. Lehrer, Ehud & Rosenberg, Dinah, 2010. "A note on the evaluation of information in zero-sum repeated games," Journal of Mathematical Economics, Elsevier, vol. 46(4), pages 393-399, July.
    17. Edward J. Sondik, 1978. "The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs," Operations Research, INFORMS, vol. 26(2), pages 282-304, April.
    18. Jun Zhuang & Vicki M. Bier, 2010. "Reasons for Secrecy and Deception in Homeland‐Security Resource Allocation," Risk Analysis, John Wiley & Sons, vol. 30(12), pages 1737-1743, December.
    19. Kamien, Morton I. & Tauman, Yair & Zamir, Shmuel, 1990. "On the value of information in a strategic conflict," Games and Economic Behavior, Elsevier, vol. 2(2), pages 129-153, June.
    20. Chu, Wai Hung Julius & Lee, Ching Chyi, 2006. "Strategic information sharing in a supply chain," European Journal of Operational Research, Elsevier, vol. 174(3), pages 1567-1579, November.
    21. Bernard De Meyer & Ehud Lehrer & Dinah Rosenberg, 2010. "Evaluating Information in Zero-Sum Games with Incomplete Information on Both Sides," Mathematics of Operations Research, INFORMS, vol. 35(4), pages 851-863, November.
    22. Zong-Zhi Lin & James C. Bean & Chelsea C. White, 2004. "A Hybrid Genetic/Optimization Algorithm for Finite-Horizon, Partially Observed Markov Decision Processes," INFORMS Journal on Computing, INFORMS, vol. 16(1), pages 27-38, February.
    23. Chelsea C. White & William T. Scherer, 1994. "Finite-Memory Suboptimal Design for Partially Observed Markov Decision Processes," Operations Research, INFORMS, vol. 42(3), pages 439-455, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Satya S. Malladi & Alan L. Erera & Chelsea C. White, 2023. "Inventory control with modulated demand and a partially observed modulation process," Annals of Operations Research, Springer, vol. 321(1), pages 343-369, February.
    2. Yanling Chang & Alan Erera & Chelsea White, 2015. "A leader–follower partially observed, multiobjective Markov game," Annals of Operations Research, Springer, vol. 235(1), pages 103-128, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yanling Chang & Alan Erera & Chelsea White, 2015. "A leader–follower partially observed, multiobjective Markov game," Annals of Operations Research, Springer, vol. 235(1), pages 103-128, December.
    2. Hao Zhang, 2010. "Partially Observable Markov Decision Processes: A Geometric Technique and Analysis," Operations Research, INFORMS, vol. 58(1), pages 214-228, February.
    3. Abhijit Gosavi, 2009. "Reinforcement Learning: A Tutorial Survey and Recent Advances," INFORMS Journal on Computing, INFORMS, vol. 21(2), pages 178-192, May.
    4. Mark Whitmeyer, 2020. "In Simple Communication Games, When Does Ex Ante Fact-Finding Benefit the Receiver?," Papers 2001.09387, arXiv.org.
    5. James T. Treharne & Charles R. Sox, 2002. "Adaptive Inventory Control for Nonstationary Demand and Partial Information," Management Science, INFORMS, vol. 48(5), pages 607-624, May.
    6. Chernonog, Tatyana & Avinadav, Tal, 2016. "A two-state partially observable Markov decision process with three actionsAuthor-Name: Ben-Zvi, Tal," European Journal of Operational Research, Elsevier, vol. 254(3), pages 957-967.
    7. Serin, Yasemin, 1995. "A nonlinear programming model for partially observable Markov decision processes: Finite horizon case," European Journal of Operational Research, Elsevier, vol. 86(3), pages 549-564, November.
    8. Yossi Aviv & Amit Pazgal, 2005. "A Partially Observed Markov Decision Process for Dynamic Pricing," Management Science, INFORMS, vol. 51(9), pages 1400-1416, September.
    9. Kloosterman, Andrew, 2015. "Public information in Markov games," Journal of Economic Theory, Elsevier, vol. 157(C), pages 28-48.
    10. Chiel van Oosterom & Lisa M. Maillart & Jeffrey P. Kharoufeh, 2017. "Optimal maintenance policies for a safety‐critical system and its deteriorating sensor," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(5), pages 399-417, August.
    11. Gossner, Olivier, 2010. "Ability and knowledge," Games and Economic Behavior, Elsevier, vol. 69(1), pages 95-106, May.
    12. Saghafian, Soroush, 2018. "Ambiguous partially observable Markov decision processes: Structural results and applications," Journal of Economic Theory, Elsevier, vol. 178(C), pages 1-35.
    13. Fabien Gensbittel, 2015. "Extensions of the Cav( u ) Theorem for Repeated Games with Incomplete Information on One Side," Mathematics of Operations Research, INFORMS, vol. 40(1), pages 80-104, February.
    14. Jesus Rios & David Rios Insua, 2012. "Adversarial Risk Analysis for Counterterrorism Modeling," Risk Analysis, John Wiley & Sons, vol. 32(5), pages 894-915, May.
    15. V. Makis & X. Jiang, 2003. "Optimal Replacement Under Partial Observations," Mathematics of Operations Research, INFORMS, vol. 28(2), pages 382-394, May.
    16. Gensbittel, Fabien & Renault, Jérôme & Peski, Marcin, 2019. "The large space of information structures," TSE Working Papers 19-1006, Toulouse School of Economics (TSE).
    17. Shoshana Anily & Abraham Grosfeld-Nir, 2006. "An Optimal Lot-Sizing and Offline Inspection Policy in the Case of Nonrigid Demand," Operations Research, INFORMS, vol. 54(2), pages 311-323, April.
    18. Hao Zhang, 2022. "Analytical Solution to a Discrete-Time Model for Dynamic Learning and Decision Making," Management Science, INFORMS, vol. 68(8), pages 5924-5957, August.
    19. Stephen M. Gilbert & Hena M Bar, 1999. "The value of observing the condition of a deteriorating machine," Naval Research Logistics (NRL), John Wiley & Sons, vol. 46(7), pages 790-808, October.
    20. White, Chelsea C. & Cheong, Taesu, 2012. "In-transit perishable product inspection," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 48(1), pages 310-330.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:235:y:2015:i:1:p:129-153:10.1007/s10479-015-1905-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.