IDEAS home Printed from
   My bibliography  Save this paper

An Approximated Solution to Continuous-Time Stochastic Optimal Control Problems Through Markov Decision Chains


  • Jacek B. Krawczyk

    (Victoria University of Wellington)

  • Alistair Windsor

    (Victoria University of Wellington)


Strategies for constructing a Markov decision chain approximating a continuous-time finite-horizon optimal control problem are investigated. Some simple, analytically soluble, examples are treated and low computational complexity is reported. Extensions to the method and implementation are discussed. In particular, relevance of the approximated solution to a stochastic renewable resource valuation problem is examined.

Suggested Citation

  • Jacek B. Krawczyk & Alistair Windsor, 1997. "An Approximated Solution to Continuous-Time Stochastic Optimal Control Problems Through Markov Decision Chains," Computational Economics 9710001, EconWPA.
  • Handle: RePEc:wpa:wuwpco:9710001
    Note: Type of Document - LaTeX; prepared on UNIX; to print on PostScript; pages: 38 ; figures: included

    Download full text from publisher

    File URL:
    Download Restriction: no

    File URL:
    Download Restriction: no

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. Veall, Michael R, 1990. "Testing for a Global Maximum in an Econometric Context," Econometrica, Econometric Society, vol. 58(6), pages 1459-1465, November.
    2. Goffe, William L. & Ferrier, Gary D. & Rogers, John, 1994. "Global optimization of statistical functions with simulated annealing," Journal of Econometrics, Elsevier, vol. 60(1-2), pages 65-99.
    3. Goffe William L., 1996. "SIMANN: A Global Optimization Algorithm using Simulated Annealing," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 1(3), pages 1-9, October.
    4. Hoffman, Dennis L. & Schmidt, Peter, 1981. "Testing the restrictions implied by the rational expectations hypothesis," Journal of Econometrics, Elsevier, vol. 15(2), pages 265-287, February.
    5. Dorsey, Robert E & Mayer, Walter J, 1995. "Genetic Algorithms for Estimation Problems with Multiple Optima, Nondifferentiability, and Other Irregular Features," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(1), pages 53-66, January.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Azzato, Jeffrey & Krawczyk, Jacek, 2006. "SOCSol4L An improved MATLAB package for approximating the solution to a continuous-time stochastic optimal control problem," MPRA Paper 1179, University Library of Munich, Germany.
    2. Azzato, Jeffrey & Krawczyk, Jacek B & Sissons, Christopher, 2011. "On loss-avoiding lump-sum pension optimization with contingent targets," Working Paper Series 1532, Victoria University of Wellington, School of Economics and Finance.
    3. Jacek B. Krawczyk, 2000. "A Markovian Approximated Solution To A Portfolio Management Problem," Computing in Economics and Finance 2000 233, Society for Computational Economics.

    More about this item


    Approximating Markov decision chains; simple noise discretisation. Natural resource valuation.;

    JEL classification:

    • C8 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • D92 - Microeconomics - - Micro-Based Behavioral Economics - - - Intertemporal Firm Choice, Investment, Capacity, and Financing
    • Q25 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Renewable Resources and Conservation - - - Water


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wpa:wuwpco:9710001. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (EconWPA). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.