IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1611.01767.html
   My bibliography  Save this paper

EM Algorithm and Stochastic Control in Economics

Author

Listed:
  • Steven Kou
  • Xianhua Peng
  • Xingbo Xu

Abstract

Generalising the idea of the classical EM algorithm that is widely used for computing maximum likelihood estimates, we propose an EM-Control (EM-C) algorithm for solving multi-period finite time horizon stochastic control problems. The new algorithm sequentially updates the control policies in each time period using Monte Carlo simulation in a forward-backward manner; in other words, the algorithm goes forward in simulation and backward in optimization in each iteration. Similar to the EM algorithm, the EM-C algorithm has the monotonicity of performance improvement in each iteration, leading to good convergence properties. We demonstrate the effectiveness of the algorithm by solving stochastic control problems in the monopoly pricing of perishable assets and in the study of real business cycle.

Suggested Citation

  • Steven Kou & Xianhua Peng & Xingbo Xu, 2016. "EM Algorithm and Stochastic Control in Economics," Papers 1611.01767, arXiv.org.
    • Handle: RePEc:arx:papers:1611.01767
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1611.01767
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christiano, Lawrence J, 1990. "Linear-Quadratic Approximation and Value-Function Iteration: A Comparison," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 99-113, January.
    2. David B. Brown & James E. Smith, 2014. "Information Relaxations, Duality, and Convex Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 62(6), pages 1394-1415, December.
    3. Guillermo Gallego & Garrett van Ryzin, 1994. "Optimal Dynamic Pricing of Inventories with Stochastic Demand over Finite Horizons," Management Science, INFORMS, vol. 40(8), pages 999-1020, August.
    4. Idris Kharroubi & Nicolas Langren'e & Huy^en Pham, 2013. "A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization," Papers 1311.4503, arXiv.org.
    5. repec:dau:papers:123456789/5524 is not listed on IDEAS
    6. David B. Brown & James E. Smith & Peng Sun, 2010. "Information Relaxations and Duality in Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 785-801, August.
    7. repec:dau:papers:123456789/5522 is not listed on IDEAS
    8. repec:cdl:anderf:qt43n1k4jb is not listed on IDEAS
    9. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," The Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    10. Hansen, Gary D., 1985. "Indivisible labor and the business cycle," Journal of Monetary Economics, Elsevier, vol. 16(3), pages 309-327, November.
    11. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, December.
    12. Guillermo Gallego & Garrett van Ryzin, 1997. "A Multiproduct Dynamic Pricing Problem and Its Applications to Network Yield Management," Operations Research, INFORMS, vol. 45(1), pages 24-41, February.
    13. Bouchard, Bruno & Touzi, Nizar, 2004. "Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations," Stochastic Processes and their Applications, Elsevier, vol. 111(2), pages 175-206, June.
    14. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    15. Long, John B, Jr & Plosser, Charles I, 1983. "Real Business Cycles," Journal of Political Economy, University of Chicago Press, vol. 91(1), pages 39-69, February.
    16. Idris Kharroubi & Nicolas Langrené & Huyên Pham, 2013. "A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization," Working Papers hal-00905899, HAL.
    17. Lars Peter Hansen & Thomas J. Sargent, 2013. "Recursive Models of Dynamic Linear Economies," Economics Books, Princeton University Press, edition 1, number 10141.
    18. Mark Broadie & Deniz Cicek & Assaf Zeevi, 2011. "General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm," Operations Research, INFORMS, vol. 59(5), pages 1211-1224, October.
    19. Broadie, Mark & Glasserman, Paul, 1997. "Pricing American-style securities using simulation," Journal of Economic Dynamics and Control, Elsevier, vol. 21(8-9), pages 1323-1352, June.
    20. Crisan, D. & Manolarakis, K. & Touzi, N., 2010. "On the Monte Carlo simulation of BSDEs: An improvement on the Malliavin weights," Stochastic Processes and their Applications, Elsevier, vol. 120(7), pages 1133-1158, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Josef Teichmann & Hanna Wutte, 2023. "Machine Learning-powered Pricing of the Multidimensional Passport Option," Papers 2307.14887, arXiv.org.
    2. Longbing Cao, 2021. "AI in Finance: Challenges, Techniques and Opportunities," Papers 2107.09051, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xianhua Peng & Steven Kou & Lekang Zhang, 2024. "A Machine Learning Algorithm for Finite-Horizon Stochastic Control Problems in Economics," Papers 2411.08668, arXiv.org, revised Dec 2024.
    2. Fernández-Villaverde, J. & Rubio-Ramírez, J.F. & Schorfheide, F., 2016. "Solution and Estimation Methods for DSGE Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 527-724, Elsevier.
    3. Dorofeenko, Victor & Lee, Gabriel S. & Salyer, Kevin D., 2010. "A new algorithm for solving dynamic stochastic macroeconomic models," Journal of Economic Dynamics and Control, Elsevier, vol. 34(3), pages 388-403, March.
    4. Alessio Trivella & Danial Mohseni-Taheri & Selvaprabu Nadarajah, 2023. "Meeting Corporate Renewable Power Targets," Management Science, INFORMS, vol. 69(1), pages 491-512, January.
    5. Aiyagari, S. Rao & Christiano, Lawrence J. & Eichenbaum, Martin, 1992. "The output, employment, and interest rate effects of government consumption," Journal of Monetary Economics, Elsevier, vol. 30(1), pages 73-86, October.
    6. Miroljub Labus & Milica Labus, 2019. "Monetary Transmission Channels in DSGE Models: Decomposition of Impulse Response Functions Approach," Computational Economics, Springer;Society for Computational Economics, vol. 53(1), pages 27-50, January.
    7. Mark Broadie & Weiwei Shen, 2017. "Numerical solutions to dynamic portfolio problems with upper bounds," Computational Management Science, Springer, vol. 14(2), pages 215-227, April.
    8. Alfonso Novales & Javier J. PÈrez, 2004. "Is It Worth Refining Linear Approximations to Non-Linear Rational Expectations Models?," Computational Economics, Springer;Society for Computational Economics, vol. 23(4), pages 343-377, June.
    9. Andrei Polbin & Sergey Drobyshevsky, 2014. "Developing a Dynamic Stochastic Model of General Equilibrium for the Russian Economy," Research Paper Series, Gaidar Institute for Economic Policy, issue 166P, pages 156-156.
    10. Guang Yang, 2009. "Local government expenditure, RBC model and regional business cycle in China-Take Tianjin for example," Frontiers of Economics in China, Springer;Higher Education Press, vol. 4(4), pages 588-600, December.
    11. John Stachurski, 2009. "Economic Dynamics: Theory and Computation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262012774, December.
    12. Khan, Aubhik & Thomas, Julia K., 2003. "Nonconvex factor adjustments in equilibrium business cycle models: do nonlinearities matter?," Journal of Monetary Economics, Elsevier, vol. 50(2), pages 331-360, March.
    13. Steve Ambler & Emanuela Cardia, 1995. "Les modèles réels de la transmission internationale du cycle économique," L'Actualité Economique, Société Canadienne de Science Economique, vol. 71(2), pages 193-217.
    14. Campbell, John Y., 1994. "Inspecting the mechanism: An analytical approach to the stochastic growth model," Journal of Monetary Economics, Elsevier, vol. 33(3), pages 463-506, June.
    15. Aruoba, S. Boragan & Fernandez-Villaverde, Jesus & Rubio-Ramirez, Juan F., 2006. "Comparing solution methods for dynamic equilibrium economies," Journal of Economic Dynamics and Control, Elsevier, vol. 30(12), pages 2477-2508, December.
    16. Aleksandar Vasilev, 2017. "A Real-Business-Cycle Model with Efficiency Wages and a Government Sector: The Case of Bulgaria," Central European Journal of Economic Modelling and Econometrics, Central European Journal of Economic Modelling and Econometrics, vol. 9(4), pages 359-377, December.
    17. Sergio Rebelo, 2005. "Real Business Cycle Models: Past, Present, and Future," NBER Working Papers 11401, National Bureau of Economic Research, Inc.
    18. Cho, Jang-Ok, 1995. "Ex post heterogeneity and the business cycle," Journal of Economic Dynamics and Control, Elsevier, vol. 19(3), pages 533-551, April.
    19. Zuzana Janko, 2007. "Nominal Wage Contracts And The Monetary Transmission Mechanism," Economic Inquiry, Western Economic Association International, vol. 45(1), pages 121-130, January.
    20. Doshchyn, Artur & Giommetti, Nicola, 2013. "Learning, Expectations, and Endogenous Business Cycles," MPRA Paper 49617, University Library of Munich, Germany.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1611.01767. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.