IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1611.01767.html
   My bibliography  Save this paper

EM Algorithm and Stochastic Control in Economics

Author

Listed:
  • Steven Kou
  • Xianhua Peng
  • Xingbo Xu

Abstract

Generalising the idea of the classical EM algorithm that is widely used for computing maximum likelihood estimates, we propose an EM-Control (EM-C) algorithm for solving multi-period finite time horizon stochastic control problems. The new algorithm sequentially updates the control policies in each time period using Monte Carlo simulation in a forward-backward manner; in other words, the algorithm goes forward in simulation and backward in optimization in each iteration. Similar to the EM algorithm, the EM-C algorithm has the monotonicity of performance improvement in each iteration, leading to good convergence properties. We demonstrate the effectiveness of the algorithm by solving stochastic control problems in the monopoly pricing of perishable assets and in the study of real business cycle.

Suggested Citation

  • Steven Kou & Xianhua Peng & Xingbo Xu, 2016. "EM Algorithm and Stochastic Control in Economics," Papers 1611.01767, arXiv.org.
    • Handle: RePEc:arx:papers:1611.01767
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1611.01767
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christiano, Lawrence J, 1990. "Linear-Quadratic Approximation and Value-Function Iteration: A Comparison," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 99-113, January.
    2. David B. Brown & James E. Smith, 2014. "Information Relaxations, Duality, and Convex Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 62(6), pages 1394-1415, December.
    3. Guillermo Gallego & Garrett van Ryzin, 1994. "Optimal Dynamic Pricing of Inventories with Stochastic Demand over Finite Horizons," Management Science, INFORMS, vol. 40(8), pages 999-1020, August.
    4. Idris Kharroubi & Nicolas Langren'e & Huy^en Pham, 2013. "A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization," Papers 1311.4503, arXiv.org.
    5. repec:dau:papers:123456789/5524 is not listed on IDEAS
    6. David B. Brown & James E. Smith & Peng Sun, 2010. "Information Relaxations and Duality in Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 785-801, August.
    7. repec:dau:papers:123456789/5522 is not listed on IDEAS
    8. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
    9. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," The Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    10. Hansen, Gary D., 1985. "Indivisible labor and the business cycle," Journal of Monetary Economics, Elsevier, vol. 16(3), pages 309-327, November.
    11. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, December.
    12. Guillermo Gallego & Garrett van Ryzin, 1997. "A Multiproduct Dynamic Pricing Problem and Its Applications to Network Yield Management," Operations Research, INFORMS, vol. 45(1), pages 24-41, February.
    13. Bouchard, Bruno & Touzi, Nizar, 2004. "Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations," Stochastic Processes and their Applications, Elsevier, vol. 111(2), pages 175-206, June.
    14. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    15. Long, John B, Jr & Plosser, Charles I, 1983. "Real Business Cycles," Journal of Political Economy, University of Chicago Press, vol. 91(1), pages 39-69, February.
    16. Idris Kharroubi & Nicolas Langrené & Huyên Pham, 2013. "A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization," Working Papers hal-00905899, HAL.
    17. Lars Peter Hansen & Thomas J. Sargent, 2013. "Recursive Models of Dynamic Linear Economies," Economics Books, Princeton University Press, edition 1, number 10141.
    18. Mark Broadie & Deniz Cicek & Assaf Zeevi, 2011. "General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm," Operations Research, INFORMS, vol. 59(5), pages 1211-1224, October.
    19. Broadie, Mark & Glasserman, Paul, 1997. "Pricing American-style securities using simulation," Journal of Economic Dynamics and Control, Elsevier, vol. 21(8-9), pages 1323-1352, June.
    20. Crisan, D. & Manolarakis, K. & Touzi, N., 2010. "On the Monte Carlo simulation of BSDEs: An improvement on the Malliavin weights," Stochastic Processes and their Applications, Elsevier, vol. 120(7), pages 1133-1158, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Josef Teichmann & Hanna Wutte, 2023. "Machine Learning-powered Pricing of the Multidimensional Passport Option," Papers 2307.14887, arXiv.org.
    2. Longbing Cao, 2021. "AI in Finance: Challenges, Techniques and Opportunities," Papers 2107.09051, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alessio Trivella & Danial Mohseni-Taheri & Selvaprabu Nadarajah, 2023. "Meeting Corporate Renewable Power Targets," Management Science, INFORMS, vol. 69(1), pages 491-512, January.
    2. Mark Broadie & Weiwei Shen, 2017. "Numerical solutions to dynamic portfolio problems with upper bounds," Computational Management Science, Springer, vol. 14(2), pages 215-227, April.
    3. Fernández-Villaverde, J. & Rubio-Ramírez, J.F. & Schorfheide, F., 2016. "Solution and Estimation Methods for DSGE Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 527-724, Elsevier.
    4. Dragos Florin Ciocan & Velibor V. Mišić, 2022. "Interpretable Optimal Stopping," Management Science, INFORMS, vol. 68(3), pages 1616-1638, March.
    5. Ludovic Gouden`ege & Andrea Molent & Antonino Zanette, 2019. "Variance Reduction Applied to Machine Learning for Pricing Bermudan/American Options in High Dimension," Papers 1903.11275, arXiv.org, revised Dec 2019.
    6. Bally Vlad & Caramellino Lucia & Zanette Antonino, 2005. "Pricing and hedging American options by Monte Carlo methods using a Malliavin calculus approach," Monte Carlo Methods and Applications, De Gruyter, vol. 11(2), pages 97-133, June.
    7. Aiyagari, S. Rao & Christiano, Lawrence J. & Eichenbaum, Martin, 1992. "The output, employment, and interest rate effects of government consumption," Journal of Monetary Economics, Elsevier, vol. 30(1), pages 73-86, October.
    8. Dorofeenko, Victor & Lee, Gabriel S. & Salyer, Kevin D., 2010. "A new algorithm for solving dynamic stochastic macroeconomic models," Journal of Economic Dynamics and Control, Elsevier, vol. 34(3), pages 388-403, March.
    9. Alfonso Novales & Javier J. PÈrez, 2004. "Is It Worth Refining Linear Approximations to Non-Linear Rational Expectations Models?," Computational Economics, Springer;Society for Computational Economics, vol. 23(4), pages 343-377, June.
    10. Juri Hinz & Tanya Tarnopolskaya & Jeremy Yee, 2020. "Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations," Annals of Operations Research, Springer, vol. 286(1), pages 583-615, March.
    11. Miroljub Labus & Milica Labus, 2019. "Monetary Transmission Channels in DSGE Models: Decomposition of Impulse Response Functions Approach," Computational Economics, Springer;Society for Computational Economics, vol. 53(1), pages 27-50, January.
    12. Moez Mrad & Nizar Touzi & Amina Zeghal, 2006. "Monte Carlo Estimation of a Joint Density Using Malliavin Calculus, and Application to American Options," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 497-531, June.
    13. Simon Scheidegger & Adrien Treccani, 2021. "Pricing American Options under High-Dimensional Models with Recursive Adaptive Sparse Expectations [Telling from Discrete Data Whether the Underlying Continuous-Time Model Is a Diffusion]," Journal of Financial Econometrics, Oxford University Press, vol. 19(2), pages 258-290.
    14. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    15. Frank Bosserhoff & An Chen & Nils Sorensen & Mitja Stadje, 2021. "On the Investment Strategies in Occupational Pension Plans," Papers 2104.08956, arXiv.org.
    16. Sebastian Becker & Patrick Cheridito & Arnulf Jentzen & Timo Welti, 2019. "Solving high-dimensional optimal stopping problems using deep learning," Papers 1908.01602, arXiv.org, revised Aug 2021.
    17. Kevin E. Beaubrun-Diant & Julien Matheron, 2008. "Rentabilités d'actifs et fluctuations économiques : une perspective d'équilibre général dynamique et stochastique," Economie & Prévision, La Documentation Française, vol. 0(2), pages 35-63.
    18. O. Samimi & Z. Mardani & S. Sharafpour & F. Mehrdoust, 2017. "LSM Algorithm for Pricing American Option Under Heston–Hull–White’s Stochastic Volatility Model," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 173-187, August.
    19. Giorgia Callegaro & Alessandro Gnoatto & Martino Grasselli, 2021. "A Fully Quantization-based Scheme for FBSDEs," Working Papers 07/2021, University of Verona, Department of Economics.
    20. Thomas Harjes, 1997. "Real business cycles in an open economy: An application to Germany," Review of World Economics (Weltwirtschaftliches Archiv), Springer;Institut für Weltwirtschaft (Kiel Institute for the World Economy), vol. 133(4), pages 635-656, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1611.01767. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.