The Markov Chain Approximation Approach for Numerical Solution of Stochastic Control Problems: Experiences from Merton's Problem

Many problems in modern financial economics involve the solution of continuous-time, continuous-state stochastic control problems. Since explicit solutions of such problems are extremely rare, efficient numerical methods are called for. The Markov chain approximation approach provides a class of methods that are simple to understand and implement. In this paper, we compare the performance of different variations of the approach on a problem with a well-known solution, namely Merton's consumption/portfolio problem. We suggest a variant of the method, which outperforms the known variants, at least when applied to this specific problem. We document that the size of the contraction parameter of the control problem is of great importance for the accuracy of the numerical results. We also demonstrate that the Richardson extrapolation technique can improce accuracy significantly.