IDEAS home Printed from https://ideas.repec.org/a/kap/compec/v56y2020i1d10.1007_s10614-019-09949-0.html
   My bibliography  Save this article

OPTCON3: An Active Learning Control Algorithm for Nonlinear Quadratic Stochastic Problems

Author

Listed:
  • V. Blueschke-Nikolaeva

    (University of Klagenfurt)

  • D. Blueschke

    (University of Klagenfurt)

  • R. Neck

    (University of Klagenfurt)

Abstract

In this paper, we describe the new OPTCON3 algorithm, which serves to determine approximately optimal policies for stochastic control problems with a quadratic objective function and nonlinear dynamic models. It includes active learning and the dual effect of optimizing policies, whereby optimal policies are used to learn about the stochastics of the dynamic system in addition to their immediate effect on the performance of the system. The OPTCON3 algorithm approximates the nonlinear model with a time-varying linear model and applies a procedure similar to that of Kendrick to the series of linearized models to calculate approximately optimal policies. The results for two simple economic models serve to test the OPTCON3 algorithm and compare it to previous solutions of the stochastic control problem. Initial evaluations show that the OPTCON3 approach may be promising to enhance our understanding of the adaptive economic policy problem under uncertainty.

Suggested Citation

  • V. Blueschke-Nikolaeva & D. Blueschke & R. Neck, 2020. "OPTCON3: An Active Learning Control Algorithm for Nonlinear Quadratic Stochastic Problems," Computational Economics, Springer;Society for Computational Economics, vol. 56(1), pages 145-162, June.
    • Handle: RePEc:kap:compec:v:56:y:2020:i:1:d:10.1007_s10614-019-09949-0
    DOI: 10.1007/s10614-019-09949-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10614-019-09949-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10614-019-09949-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Blueschke-Nikolaeva, V. & Blueschke, D. & Neck, R., 2012. "Optimal control of nonlinear dynamic econometric models: An algorithm and an application," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3230-3240.
    2. D. Blueschke & I. Savin & V. Blueschke-Nikolaeva, 2020. "An Evolutionary Approach to Passive Learning in Optimal Control Problems," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 659-673, October.
    3. Kendrick, David, 1982. "Caution and probing in a macroeconomic model," Journal of Economic Dynamics and Control, Elsevier, vol. 4(1), pages 149-170, November.
    4. Mizrach, Bruce, 1991. "Nonconvexities in a stochastic control problem with learning," Journal of Economic Dynamics and Control, Elsevier, vol. 15(3), pages 515-538, July.
    5. Elizabeth Chase MacRae, 1972. "Linear Decision with Experimentation," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 1, number 4, pages 437-447, National Bureau of Economic Research, Inc.
    6. Amman, Hans M & Kendrick, David A, 1995. "Nonconvexities in Stochastic Control Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 36(2), pages 455-475, May.
    7. Yaakov Bar-Shalom & Edison Tse, 1976. "Caution, Probing, and the Value of Information in the Control of Uncertain Systems," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 3, pages 323-337, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    2. Kendrick, David A., 2005. "Stochastic control for economic models: past, present and the paths ahead," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 3-30, January.
    3. Marco Tucci, 2006. "Understanding the Difference Between Robust Control and Optimal Control in a Linear Discrete-Time System with Time-Varying Parameters," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 533-558, June.
    4. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    5. Tucci, Marco P. & Kendrick, David A. & Amman, Hans M., 2010. "The parameter set in an adaptive control Monte Carlo experiment: Some considerations," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1531-1549, September.
    6. Wieland, Volker, 2000. "Learning by doing and the value of optimal experimentation," Journal of Economic Dynamics and Control, Elsevier, vol. 24(4), pages 501-534, April.
    7. David Kendrick & Hans Amman, 2006. "A Classification System for Economic Stochastic Control Models," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 453-481, June.
    8. repec:use:tkiwps:2020 is not listed on IDEAS
    9. Tucci, Marco P., 2002. "A note on global optimization in adaptive control, econometrics and macroeconomics," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1739-1764, August.
    10. Volker Wieland, "undated". "Monetary Policy and Uncertainty about the Natural Unemployment Rate," Computing in Economics and Finance 1997 11, Society for Computational Economics.
    11. In Chang Hwang & Richard S. J. Tol & Marjan W. Hofkes, 2019. "Active Learning and Optimal Climate Policy," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 73(4), pages 1237-1264, August.
    12. Bond, Craig A., 2008. "On the Potential Use of Adaptive Control Methods for Improving Adaptive Natural Resource Management," Working Papers 108721, Colorado State University, Department of Agricultural and Resource Economics.
    13. D.A. Kendrick & H.M. Amman & M.P. Tucci, 2008. "Learning About Learning in Dynamic Economic Models," Working Papers 08-20, Utrecht School of Economics.
    14. Amman, Hans M & Kendrick, David A, 1999. "Should Macroeconomic Policy Makers Consider Parameter Covariances?," Computational Economics, Springer;Society for Computational Economics, vol. 14(3), pages 263-267, December.
    15. D. Blueschke & V. Blueschke-Nikolaeva & R. Neck, 2013. "Stochastic Control of Linear and Nonlinear Econometric Models: Some Computational Aspects," Computational Economics, Springer;Society for Computational Economics, vol. 42(1), pages 107-118, June.
    16. Tim Willems, 2017. "Actively Learning by Pricing: A Model of an Experimenting Seller," Economic Journal, Royal Economic Society, vol. 127(604), pages 2216-2239, September.
    17. H.M. Amman & D.A. Kendrick, 2012. "Conjectures on the policy function in the presence of optimal experimentation," Working Papers 12-09, Utrecht School of Economics.
    18. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    19. David Kendrick, 1976. "Applications of Control Theory to Macroeconomics," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 2, pages 171-190, National Bureau of Economic Research, Inc.
    20. In Chang Hwang, 2016. "Active learning and optimal climate policy," EcoMod2016 9611, EcoMod.
    21. Dmitri Blueschke & Ivan Savin, 2015. "No such thing like perfect hammer: comparing different objective function specifications for optimal control," Jena Economics Research Papers 2015-005, Friedrich-Schiller-University Jena.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:compec:v:56:y:2020:i:1:d:10.1007_s10614-019-09949-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.