# A Malliavin-based Monte-Carlo Approach for Numerical Solution of Stochastic Control Problems: Experiences from Merton's Problem

## Author Info

• Simon Lysbjerg Hansen

()
(Accounting and Finance University of Southern Denmark)

## Abstract

The problem of choosing optimal investment and consumption strategies has been widely studied. In continuous time theory the pioneering work by Merton (1969) is a standard reference. In his work, Merton studied a continuous time economy with constant investment opportunities. Since then Merton's problem has been extended in many ways to capture empirically observed investment and consumption behavior. As more realism is incorporated into a model, the problem of optimal investment and consumption becomes harder to solve. Only rarely can analytical solutions be found, and only for problems possessing nice characteristics. To solve problems lacking analytical solutions we must apply numerical methods. Many realistic problems, however, are difficult to solve even numerically, due to their dimensionality. The purpose of this paper is to present a numerical procedure for solving high-dimensional stochastic control problems arising in the study of optimal portfolio choice. For expositional reasons we develop the algorithm in one dimension, but the mathematical results needed can be generalized to a multi-dimensional setting. The starting point of the algorithm is an initial guess about the agent's investment and consumption strategies at all times and wealth levels. Given this guess it is possible to simulate the wealth process until the investment horizon of the agent. We exploit the dynamic programming principle to break the problem into a series of smaller one-period problems, which can be solved recursively backwards. To be specific we determine first-order conditions relating the optimal controls to the value function in the next period. Starting from the final date we now numerically solve the first-order conditions for all simulated paths iteratively backwards. The investment and consumption strategies resulting from this procedure are used to update the simulated wealth paths, and the procedure can be repeated until it converges. The numerical properties of the algorithm are analyzed by testing it on Merton's optimal portfolio choice problem. The reason for this is that the solution to Merton's problem is explicitly known and can therefore serve as a benchmark for the algorithm. Our results indicate that it is possible to obtain some sort of convergence for both the initial controls and the distribution of their future values. Bearing in mind that we intend to apply the algorithm to a multi-dimensional setting, we also consider the possible complications that might arise. However, the state variables added will in most cases be exogenous non-controllable processes, which does not complicate the optimization routine in the proposed algorithm. Problems with computer storage could arise, but they should be solvable with clever computer programming

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.sam.sdu.dk/~slh/papers/mmc.pdf
File Function: main text

## Bibliographic Info

Paper provided by Society for Computational Economics in its series Computing in Economics and Finance 2005 with number 391.

as in new window
Length:
Date of revision:
Handle: RePEc:sce:scecf5:391

Contact details of provider:
Email:
Web page: http://comp-econ.org/

## Related research

Keywords:

Find related papers by JEL classification:

• C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
• G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions

This paper has been announced in the following NEP Reports:

## References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
1. John Campbell & Joao F. Cocco, 2002. "Household Risk Management and Optimal Mortgage Choice," Computing in Economics and Finance 2002 47, Society for Computational Economics.
2. Bouchard, Bruno & Touzi, Nizar & Ekeland, Ivar, 2004. "On the Malliavin approach to Monte Carlo approximation of conditional expectations," Economics Papers from University Paris Dauphine 123456789/1802, Paris Dauphine University.
3. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
4. Bruno Bouchard & Ivar Ekeland & Nizar Touzi, 2004. "On the Malliavin approach to Monte Carlo approximation of conditional expectations," Finance and Stochastics, Springer, vol. 8(1), pages 45-71, January.
5. Longstaff, Francis A, 2001. "Optimal Portfolio Choice and the Valuation of Illiquid Securities," Review of Financial Studies, Society for Financial Studies, vol. 14(2), pages 407-31.
6. Michael W. Brandt & Amit Goyal & Pedro Santa-Clara & Jonathan Storud, 2004. "A Simulation Approach to Dynamic Portfolio Choice with an Application to Learning About Return Predictability," NBER Working Papers 10934, National Bureau of Economic Research, Inc.
7. Claus Munk, 1998. "The Markov Chain Approximation Approach for Numerical Solution of Stochastic Control Problems: Experiences from Merton's Problem," Finance 9802002, EconWPA.
8. Merton, Robert C, 1969. "Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case," The Review of Economics and Statistics, MIT Press, vol. 51(3), pages 247-57, August.
9. Jér�me B. Detemple & René Garcia & Marcel Rindisbacher, 2003. "A Monte Carlo Method for Optimal Portfolios," Journal of Finance, American Finance Association, vol. 58(1), pages 401-446, 02.
10. Samuelson, Paul A, 1969. "Lifetime Portfolio Selection by Dynamic Stochastic Programming," The Review of Economics and Statistics, MIT Press, vol. 51(3), pages 239-46, August.
11. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-47.
Full references (including those not matched with items on IDEAS)

## Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

## Corrections

When requesting a correction, please mention this item's handle: RePEc:sce:scecf5:391. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F. Baum).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.