IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v219y2014i1p63-8410.1007-s10479-012-1220-4.html
   My bibliography  Save this article

Markov Limid processes for representing and solving renewal problems

Author

Listed:
  • Erik Jørgensen
  • Anders Kristensen
  • Dennis Nilsson

Abstract

In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams ( Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept. Copyright Springer Science+Business Media, LLC 2014

Suggested Citation

  • Erik Jørgensen & Anders Kristensen & Dennis Nilsson, 2014. "Markov Limid processes for representing and solving renewal problems," Annals of Operations Research, Springer, vol. 219(1), pages 63-84, August.
    • Handle: RePEc:spr:annopr:v:219:y:2014:i:1:p:63-84:10.1007/s10479-012-1220-4
    DOI: 10.1007/s10479-012-1220-4
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-012-1220-4
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-012-1220-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Steffen L. Lauritzen & Dennis Nilsson, 2001. "Representing and Solving Decision Problems with Limited Information," Management Science, INFORMS, vol. 47(9), pages 1235-1251, September.
    2. Prakash P. Shenoy, 1992. "Valuation-Based Systems for Bayesian Decision Analysis," Operations Research, INFORMS, vol. 40(3), pages 463-484, June.
    3. Mourits, M. C. M. & Huirne, R. B. M. & Dijkhuizen, A. A. & Kristensen, A. R. & Galligan, D. T., 1999. "Economic optimization of dairy heifer management decisions," Agricultural Systems, Elsevier, vol. 61(1), pages 17-31, July.
    4. Kristensen, Anders Ringgaard, 1993. "Bayesian Updating in Hierarchic Markov Processes Applied to the Animal Replacement Problem," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 20(2), pages 223-239.
    5. Anders Kristensen & Erik Jørgensen, 2000. "Multi‐level hierarchic Markov processes as a framework for herd management support," Annals of Operations Research, Springer, vol. 94(1), pages 69-89, January.
    6. James C. Cox, 1998. "Quantifying the Effects of Sow-Herd Management Information Systems on Farmers' Decision Making Using Experimental Economics," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 80(4), pages 821-829.
    7. Kristensen, Anders R., 1988. "Hierarchic Markov processes and their applications in replacement models," European Journal of Operational Research, Elsevier, vol. 35(2), pages 207-215, May.
    8. Ross D. Shachter, 1986. "Evaluating Influence Diagrams," Operations Research, INFORMS, vol. 34(6), pages 871-882, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Salo, Ahti & Andelmin, Juho & Oliveira, Fabricio, 2022. "Decision programming for mixed-integer multi-stage optimization under uncertainty," European Journal of Operational Research, Elsevier, vol. 299(2), pages 550-565.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bielza, Concha & Gómez, Manuel & Shenoy, Prakash P., 2011. "A review of representation issues and modeling challenges with influence diagrams," Omega, Elsevier, vol. 39(3), pages 227-241, June.
    2. Borgonovo, Emanuele & Tonoli, Fabio, 2014. "Decision-network polynomials and the sensitivity of decision-support models," European Journal of Operational Research, Elsevier, vol. 239(2), pages 490-503.
    3. Lars Relund Nielsen & Erik Jørgensen & Søren Højsgaard, 2011. "Embedding a state space model into a Markov decision process," Annals of Operations Research, Springer, vol. 190(1), pages 289-309, October.
    4. Koller, Daphne & Milch, Brian, 2003. "Multi-agent influence diagrams for representing and solving games," Games and Economic Behavior, Elsevier, vol. 45(1), pages 181-221, October.
    5. Finn Jensen & Thomas Nielsen, 2013. "Probabilistic decision graphs for optimization under uncertainty," Annals of Operations Research, Springer, vol. 204(1), pages 223-248, April.
    6. Cobb, Barry R. & Shenoy, Prakash P., 2008. "Decision making with hybrid influence diagrams using mixtures of truncated exponentials," European Journal of Operational Research, Elsevier, vol. 186(1), pages 261-275, April.
    7. Apiruk Detwarasiti & Ross D. Shachter, 2005. "Influence Diagrams for Team Decision Analysis," Decision Analysis, INFORMS, vol. 2(4), pages 207-228, December.
    8. Prakash Shenoy, 1998. "Game Trees For Decision Analysis," Theory and Decision, Springer, vol. 44(2), pages 149-171, April.
    9. Concha Bielza & Prakash P. Shenoy, 1999. "A Comparison of Graphical Techniques for Asymmetric Decision Problems," Management Science, INFORMS, vol. 45(11), pages 1552-1569, November.
    10. Nielsen, Lars Relund & Kristensen, Anders Ringgaard, 2006. "Finding the K best policies in a finite-horizon Markov decision process," European Journal of Operational Research, Elsevier, vol. 175(2), pages 1164-1179, December.
    11. Barry R. Cobb, 2007. "Influence Diagrams with Continuous Decision Variables and Non-Gaussian Uncertainties," Decision Analysis, INFORMS, vol. 4(3), pages 136-155, September.
    12. Misuri, Alessio & Khakzad, Nima & Reniers, Genserik & Cozzani, Valerio, 2019. "A Bayesian network methodology for optimal security management of critical infrastructures," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    13. Francisco Javier Díez & Manuel Arias & Jorge Pérez-Martín & Manuel Luque, 2022. "Teaching Probabilistic Graphical Models with OpenMarkov," Mathematics, MDPI, vol. 10(19), pages 1-20, September.
    14. Steffen L. Lauritzen & Dennis Nilsson, 2001. "Representing and Solving Decision Problems with Limited Information," Management Science, INFORMS, vol. 47(9), pages 1235-1251, September.
    15. Yates, C.M. & Rehman, T., 1998. "A linear programming formulation of the Markovian decision process approach to modelling the dairy replacement problem," Agricultural Systems, Elsevier, vol. 58(2), pages 185-201, October.
    16. Barry R. Cobb, 2021. "Statistical Process Control for the Number of Defectives with Limited Memory," Decision Analysis, INFORMS, vol. 18(3), pages 203-217, September.
    17. Yates, C. M. & Rehman, T. & Chamberlain, A. T., 1996. "Evaluation of the potential effects of embryo transfer on milk production on commercial dairy herds: The development of a markov chain model," Agricultural Systems, Elsevier, vol. 50(1), pages 65-79.
    18. Fernandez del Pozo, J. A. & Bielza, C. & Gomez, M., 2005. "A list-based compact representation for large decision tables management," European Journal of Operational Research, Elsevier, vol. 160(3), pages 638-662, February.
    19. Yijing Li & Prakash P. Shenoy, 2012. "A Framework for Solving Hybrid Influence Diagrams Containing Deterministic Conditional Distributions," Decision Analysis, INFORMS, vol. 9(1), pages 55-75, March.
    20. Lan Ge & Anders Kristensen & Monique Mourits & Ruud Huirne, 2014. "A new decision support framework for managing foot-and-mouth disease epidemics," Annals of Operations Research, Springer, vol. 219(1), pages 49-62, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:219:y:2014:i:1:p:63-84:10.1007/s10479-012-1220-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.