IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v58y2010i4-part-2p1235-1246.html
   My bibliography  Save this article

Myopic Solutions of Homogeneous Sequential Decision Processes

Author

Listed:
  • Matthew J. Sobel

    (Department of Operations, Weatherhead School of Management, Case Western Reserve University, Cleveland, Ohio 44106)

  • Wei Wei

    (Department of Operations, Weatherhead School of Management, Case Western Reserve University, Cleveland, Ohio 44106)

Abstract

An optimum of a Markov decision process (MDP) is myopic if it can be obtained by solving a series of static problems. Myopic optima are desirable because they can be computed relatively easily. We identify new classes of MDPs with myopic optima and sequential games with myopic equilibrium points. In one of the classes, the single-period reward is homogeneous with respect to the state variable. We illustrate the results with models of revenue management and investment.

Suggested Citation

  • Matthew J. Sobel & Wei Wei, 2010. "Myopic Solutions of Homogeneous Sequential Decision Processes," Operations Research, INFORMS, vol. 58(4-part-2), pages 1235-1246, August.
    • Handle: RePEc:inm:oropre:v:58:y:2010:i:4-part-2:p:1235-1246
    DOI: 10.1287/opre.1090.0767
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.1090.0767
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.1090.0767?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Serguei Netessine & Robert A. Shumsky, 2005. "Revenue Management Games: Horizontal and Vertical Competition," Management Science, INFORMS, vol. 51(5), pages 813-831, May.
    2. Jeffrey I. McGill & Garrett J. van Ryzin, 1999. "Revenue Management: Research Overview and Prospects," Transportation Science, INFORMS, vol. 33(2), pages 233-256, May.
    3. Nicholas C. Petruzzi & Maqbool Dada, 1999. "Pricing and the Newsvendor Problem: A Review with Extensions," Operations Research, INFORMS, vol. 47(2), pages 183-194, April.
    4. Hakansson, Nils H, 1970. "Optimal Investment and Consumption Strategies Under Risk for a Class of Utility Functions," Econometrica, Econometric Society, vol. 38(5), pages 587-607, September.
    5. Arthur F. Veinott, Jr., 1965. "Optimal Policy for a Multi-Product, Dynamic, Nonstationary Inventory Problem," Management Science, INFORMS, vol. 12(3), pages 206-222, November.
    6. Kirman, Alan P & Sobel, Matthew J, 1974. "Dynamic Oligopoly with Inventories," Econometrica, Econometric Society, vol. 42(2), pages 279-287, March.
    7. Milgrom, Paul & Roberts, John, 1994. "Comparing Equilibria," American Economic Review, American Economic Association, vol. 84(3), pages 441-459, June.
    8. Matthew J. Sobel, 1990. "Myopic Solutions of Affine Dynamic Models," Operations Research, INFORMS, vol. 38(5), pages 847-853, October.
    9. Matthew J. Sobel, 1990. "Higher-Order and Average Reward Myopic-Affine Dynamic Models," Mathematics of Operations Research, INFORMS, vol. 15(2), pages 299-310, May.
    10. Matthew J. Sobel, 1981. "Myopic Solutions of Markov Decision Processes and Stochastic Games," Operations Research, INFORMS, vol. 29(5), pages 995-1009, October.
    11. Edward Ignall & Arthur F. Veinott, Jr., 1969. "Optimality of Myopic Inventory Policies for Several Substitute Products," Management Science, INFORMS, vol. 15(5), pages 284-304, January.
    12. Alvarez, Fernando & Stokey, Nancy L., 1998. "Dynamic Programming with Homogeneous Functions," Journal of Economic Theory, Elsevier, vol. 82(1), pages 167-189, September.
    13. Harvey M. Wagner, 1960. "On the Optimality of Pure Strategies," Management Science, INFORMS, vol. 6(3), pages 268-269, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kenneth Judd & Garrett van Ryzin, 2010. "Preface to the Special Issue on Computational Economics," Operations Research, INFORMS, vol. 58(4-part-2), pages 1035-1036, August.
    2. Stoll, Sebastian & Zöttl, Gregor, 2014. "Transparency in Buyer-Determined Auctions: Should Quality be Private or Public?," Discussion Paper Series of SFB/TR 15 Governance and the Efficiency of Economic Systems 459, Free University of Berlin, Humboldt University of Berlin, University of Bonn, University of Mannheim, University of Munich.
    3. Matthew J. Sobel & Volodymyr Babich, 2012. "Optimality of Myopic Policies for Dynamic Lot-Sizing Problems in Serial Production Lines with Random Yields and Autoregressive Demand," Operations Research, INFORMS, vol. 60(6), pages 1520-1536, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jie Ning & Matthew J. Sobel, 2019. "Easy Affine Markov Decision Processes," Operations Research, INFORMS, vol. 67(6), pages 1719-1737, November.
    2. Wang, Yunzeng, 2001. "The optimality of myopic stocking policies for systems with decreasing purchasing prices," European Journal of Operational Research, Elsevier, vol. 133(1), pages 153-159, August.
    3. Lin, Kyle Y. & Sibdari, Soheil Y., 2009. "Dynamic price competition with discrete customer choices," European Journal of Operational Research, Elsevier, vol. 197(3), pages 969-980, September.
    4. Matthew J. Sobel & Volodymyr Babich, 2012. "Optimality of Myopic Policies for Dynamic Lot-Sizing Problems in Serial Production Lines with Random Yields and Autoregressive Demand," Operations Research, INFORMS, vol. 60(6), pages 1520-1536, December.
    5. Xu, Ningxiong, 2008. "Myopic policy for a two-product and multi-period supply contract with different delivery lead times and storage limitation," International Journal of Production Economics, Elsevier, vol. 115(1), pages 179-188, September.
    6. Kyparisis, George J. & Koulamas, Christos, 2018. "Optimal pricing and seat allocation for a two-cabin airline revenue management problem," International Journal of Production Economics, Elsevier, vol. 201(C), pages 18-25.
    7. Zeynep Müge Avsar & Melike Baykal‐Gürsoy, 2002. "Inventory control under substitutable demand: A stochastic game application," Naval Research Logistics (NRL), John Wiley & Sons, vol. 49(4), pages 359-375, June.
    8. Dasci, A. & Karakul, M., 2009. "Two-period dynamic versus fixed-ratio pricing in a capacity constrained duopoly," European Journal of Operational Research, Elsevier, vol. 197(3), pages 945-968, September.
    9. Hu, Qiying & Wei, Yihua & Xia, Yusen, 2010. "Revenue management for a supply chain with two streams of customers," European Journal of Operational Research, Elsevier, vol. 200(2), pages 582-598, January.
    10. Lingxiu Dong & Panos Kouvelis & Zhongjun Tian, 2009. "Dynamic Pricing and Inventory Control of Substitute Products," Manufacturing & Service Operations Management, INFORMS, vol. 11(2), pages 317-339, December.
    11. Hariga, Moncer A., 2010. "A single-item continuous review inventory problem with space restriction," International Journal of Production Economics, Elsevier, vol. 128(1), pages 153-158, November.
    12. J Pinder, 2005. "Using revenue management to improve pricing and capacity management in programme management," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(1), pages 75-87, January.
    13. Chou, Mabel & Sim, Chee-Khian & Yuan, Xue-Ming, 2013. "Optimal policies for inventory systems with two types of product sharing common hardware platforms: Single period and finite horizon," European Journal of Operational Research, Elsevier, vol. 224(2), pages 283-292.
    14. Choi, Sungyong & Ruszczynski, Andrzej, 2011. "A multi-product risk-averse newsvendor with exponential utility function," European Journal of Operational Research, Elsevier, vol. 214(1), pages 78-84, October.
    15. Jie Ning, 2021. "Reducible Markov Decision Processes and Stochastic Games," Production and Operations Management, Production and Operations Management Society, vol. 30(8), pages 2726-2751, August.
    16. Toda, Alexis Akira, 2014. "Incomplete market dynamics and cross-sectional distributions," Journal of Economic Theory, Elsevier, vol. 154(C), pages 310-348.
    17. Chao, Gary H., 2013. "Production and availability policies through the Markov Decision Process and myopic methods for contractual and selective orders," European Journal of Operational Research, Elsevier, vol. 225(3), pages 383-392.
    18. Yehuda Bassok & Ravi Anupindi & Ram Akella, 1999. "Single-Period Multiproduct Inventory Models with Substitution," Operations Research, INFORMS, vol. 47(4), pages 632-642, August.
    19. Xin, Linwei & Goldberg, David A., 2021. "Time (in)consistency of multistage distributionally robust inventory models with moment constraints," European Journal of Operational Research, Elsevier, vol. 289(3), pages 1127-1141.
    20. Petruzzi, Nicholas & Monahan, George E., 2002. "Managing Fashion Goods Inventories:," Working Papers 02-0117, University of Illinois at Urbana-Champaign, College of Business.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:58:y:2010:i:4-part-2:p:1235-1246. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.