IDEAS home Printed from
   My bibliography  Save this paper

A Malliavin-based Monte-Carlo Approach for Numerical Solution of Stochastic Control Problems: Experiences from Merton's Problem


  • Simon Lysbjerg Hansen

    () (Accounting and Finance University of Southern Denmark)


The problem of choosing optimal investment and consumption strategies has been widely studied. In continuous time theory the pioneering work by Merton (1969) is a standard reference. In his work, Merton studied a continuous time economy with constant investment opportunities. Since then Merton's problem has been extended in many ways to capture empirically observed investment and consumption behavior. As more realism is incorporated into a model, the problem of optimal investment and consumption becomes harder to solve. Only rarely can analytical solutions be found, and only for problems possessing nice characteristics. To solve problems lacking analytical solutions we must apply numerical methods. Many realistic problems, however, are difficult to solve even numerically, due to their dimensionality. The purpose of this paper is to present a numerical procedure for solving high-dimensional stochastic control problems arising in the study of optimal portfolio choice. For expositional reasons we develop the algorithm in one dimension, but the mathematical results needed can be generalized to a multi-dimensional setting. The starting point of the algorithm is an initial guess about the agent's investment and consumption strategies at all times and wealth levels. Given this guess it is possible to simulate the wealth process until the investment horizon of the agent. We exploit the dynamic programming principle to break the problem into a series of smaller one-period problems, which can be solved recursively backwards. To be specific we determine first-order conditions relating the optimal controls to the value function in the next period. Starting from the final date we now numerically solve the first-order conditions for all simulated paths iteratively backwards. The investment and consumption strategies resulting from this procedure are used to update the simulated wealth paths, and the procedure can be repeated until it converges. The numerical properties of the algorithm are analyzed by testing it on Merton's optimal portfolio choice problem. The reason for this is that the solution to Merton's problem is explicitly known and can therefore serve as a benchmark for the algorithm. Our results indicate that it is possible to obtain some sort of convergence for both the initial controls and the distribution of their future values. Bearing in mind that we intend to apply the algorithm to a multi-dimensional setting, we also consider the possible complications that might arise. However, the state variables added will in most cases be exogenous non-controllable processes, which does not complicate the optimization routine in the proposed algorithm. Problems with computer storage could arise, but they should be solvable with clever computer programming

Suggested Citation

  • Simon Lysbjerg Hansen, 2005. "A Malliavin-based Monte-Carlo Approach for Numerical Solution of Stochastic Control Problems: Experiences from Merton's Problem," Computing in Economics and Finance 2005 391, Society for Computational Economics.
  • Handle: RePEc:sce:scecf5:391

    Download full text from publisher

    File URL:
    File Function: main text
    Download Restriction: no

    References listed on IDEAS

    1. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    2. Michael W. Brandt & Amit Goyal & Pedro Santa-Clara & Jonathan R. Stroud, 2005. "A Simulation Approach to Dynamic Portfolio Choice with an Application to Learning About Return Predictability," Review of Financial Studies, Society for Financial Studies, vol. 18(3), pages 831-873.
    3. John Y. Campbell & João F. Cocco, 2003. "Household Risk Management and Optimal Mortgage Choice," The Quarterly Journal of Economics, Oxford University Press, vol. 118(4), pages 1449-1494.
    4. Jérôme B. Detemple & René Garcia & Marcel Rindisbacher, 2003. "A Monte Carlo Method for Optimal Portfolios," Journal of Finance, American Finance Association, vol. 58(1), pages 401-446, February.
    5. Longstaff, Francis A, 2001. "Optimal Portfolio Choice and the Valuation of Illiquid Securities," Review of Financial Studies, Society for Financial Studies, vol. 14(2), pages 407-431.
    6. Bruno Bouchard & Ivar Ekeland & Nizar Touzi, 2004. "On the Malliavin approach to Monte Carlo approximation of conditional expectations," Finance and Stochastics, Springer, vol. 8(1), pages 45-71, January.
    7. Claus Munk, 1998. "The Markov Chain Approximation Approach for Numerical Solution of Stochastic Control Problems: Experiences from Merton's Problem," Finance 9802002, EconWPA.
    8. repec:dau:papers:123456789/1802 is not listed on IDEAS
    9. Bénédicte Vidaillet & V. D'Estaintot & P. Abécassis, 2005. "Introduction," Post-Print hal-00287137, HAL.
    10. Paul A. Samuelson, 2011. "Lifetime Portfolio Selection by Dynamic Stochastic Programming," World Scientific Book Chapters,in: THE KELLY CAPITAL GROWTH INVESTMENT CRITERION THEORY and PRACTICE, chapter 31, pages 465-472 World Scientific Publishing Co. Pte. Ltd..
    11. Merton, Robert C, 1969. "Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case," The Review of Economics and Statistics, MIT Press, vol. 51(3), pages 247-257, August.
    12. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
    Full references (including those not matched with items on IDEAS)

    More about this item

    JEL classification:

    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sce:scecf5:391. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F. Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.