IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_9255.html
   My bibliography  Save this paper

Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm

Author

Listed:
  • Rui (Aruhan) Shi

Abstract

This exercise offers an innovative learning mechanism to model economic agent’s decision-making process using a deep reinforcement learning algorithm. In particular, this AI agent is born in an economic environment with no information on the underlying economic structure and its own preference. I model how the AI agent learns from square one in terms of how it collects and processes information. It is able to learn in real time through constantly interacting with the environment and adjusting its actions accordingly (i.e., online learning). I illustrate that the economic agent under deep reinforcement learning is adaptive to changes in a given environment in real time. AI agents differ in their ways of collecting and processing information, and this leads to different learning behaviours and welfare distinctions. The chosen economic structure can be generalised to other decision-making processes and economic models.

Suggested Citation

  • Rui (Aruhan) Shi, 2021. "Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm," CESifo Working Paper Series 9255, CESifo.
  • Handle: RePEc:ces:ceswps:_9255
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp9255.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sargent, Thomas J, 1972. "Rational Expectations and the Term Structure of Interest Rates," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 4(1), pages 74-97, Part I Fe.
    2. Marcet, Albert & Sargent, Thomas J, 1989. "Convergence of Least-Squares Learning in Environments with Hidden State Variables and Private Information," Journal of Political Economy, University of Chicago Press, vol. 97(6), pages 1306-1322, December.
    3. Arthur Charpentier & Romuald Elie & Carl Remlinger, 2020. "Reinforcement Learning in Economics and Finance," Papers 2003.10014, arXiv.org.
    4. Lucas, Robert Jr., 1972. "Expectations and the neutrality of money," Journal of Economic Theory, Elsevier, vol. 4(2), pages 103-124, April.
    5. Milton Friedman, 1957. "Introduction to "A Theory of the Consumption Function"," NBER Chapters, in: A Theory of the Consumption Function, pages 1-6, National Bureau of Economic Research, Inc.
    6. Mingli Chen & Andreas Joseph & Michael Kumhof & Xinlei Pan & Xuan Zhou, 2021. "Deep Reinforcement Learning in a Monetary Model," Papers 2104.09368, arXiv.org, revised Jan 2023.
    7. Bray, Margaret, 1982. "Learning, estimation, and the stability of rational expectations," Journal of Economic Theory, Elsevier, vol. 26(2), pages 318-339, April.
    8. George W. Evans, 2001. "Expectations in Macroeconomics. Adaptive versus Eductive Learning," Revue Économique, Programme National Persée, vol. 52(3), pages 573-582.
    9. Milton Friedman, 1957. "A Theory of the Consumption Function," NBER Books, National Bureau of Economic Research, Inc, number frie57-1, May.
    10. Amir Mosavi & Pedram Ghamisi & Yaser Faghan & Puhong Duan, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Papers 2004.01509, arXiv.org.
    11. Mosavi, Amir & Faghan, Yaser & Ghamisi, Pedram & Duan, Puhong & Ardabili, Sina Faizollahzadeh & Hassan, Salwana & Band, Shahab S., 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," OSF Preprints jrc58, Center for Open Science.
    12. Maliar, Serguei & Winant, Pablo, 2019. "Will Artificial Intelligence Replace Computational Economists Any Time Soon?," CEPR Discussion Papers 14024, C.E.P.R. Discussion Papers.
    13. Susan Athey, 2018. "The Impact of Machine Learning on Economics," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 507-547, National Bureau of Economic Research, Inc.
    14. Sargent, Thomas J., 1993. "Bounded Rationality in Macroeconomics: The Arne Ryde Memorial Lectures," OUP Catalogue, Oxford University Press, number 9780198288695.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rui & Shi, 2021. "Can an AI agent hit a moving target?," Papers 2110.02474, arXiv.org, revised Oct 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rui & Shi, 2021. "Learning from zero: how to make consumption-saving decisions in a stochastic environment with an AI algorithm," Papers 2105.10099, arXiv.org, revised Feb 2022.
    2. Rui & Shi, 2021. "Can an AI agent hit a moving target?," Papers 2110.02474, arXiv.org, revised Oct 2022.
    3. Gerunov, Anton, 2014. "Критичен Преглед На Основните Подходи За Моделиране На Икономическите Очаквания [A Critical Review of Major Approaches for Modeling Economic Expectations]," MPRA Paper 68797, University Library of Munich, Germany.
    4. Kedar-Levy, Haim, 2020. "Price discovery in the small and in the large: Momentum and reversal, bubbles, and crashes," Journal of Financial Markets, Elsevier, vol. 48(C).
    5. Hommes, Cars, 2018. "Behavioral & experimental macroeconomics and policy analysis: a complex systems approach," Working Paper Series 2201, European Central Bank.
    6. Geweke, J. & Joel Horowitz & Pesaran, M.H., 2006. "Econometrics: A Bird’s Eye View," Cambridge Working Papers in Economics 0655, Faculty of Economics, University of Cambridge.
    7. Lucas Papademos, 2005. "Macroeconomic theory and monetary policy: the contributions of Franco Modigliani and the ongoing debate," BNL Quarterly Review, Banca Nazionale del Lavoro, vol. 58(233-234), pages 187-214.
    8. Akhand Akhtar Hossain, 2009. "Central Banking and Monetary Policy in the Asia-Pacific," Books, Edward Elgar Publishing, number 12777.
    9. Aurélien Goutsmedt & Erich Pinzon-Fuchs & Matthieu Renault & Francesco Sergi, 2015. "Criticizing the Lucas Critique: Macroeconometricians' Response to Robert Lucas," Post-Print halshs-01179114, HAL.
    10. Athreya, Kartik B., 2014. "Big Ideas in Macroeconomics: A Nontechnical View," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262019736, December.
    11. Bernasconi, Michele & Kirchkamp, Oliver & Paruolo, Paolo, 2009. "Do fiscal variables affect fiscal expectations? Experiments with real world and lab data," Journal of Economic Behavior & Organization, Elsevier, vol. 70(1-2), pages 253-265, May.
    12. Felipe Pérez, 1998. "- Private Experience In Adaptive Learning Models," Working Papers. Serie AD 1998-03, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    13. Jan-Oliver Menz, 2010. "Uncertainty, social norms and consumption theory: Post and New Keynesian approaches," European Journal of Economics and Economic Policies: Intervention, Edward Elgar Publishing, vol. 7(1), pages 125-146.
    14. Bao, Te & Duffy, John, 2016. "Adaptive versus eductive learning: Theory and evidence," European Economic Review, Elsevier, vol. 83(C), pages 64-89.
    15. Farmer, Roger & Zabczyk, Pawel, 2016. "The theory of unconventional monetary policy," Bank of England working papers 613, Bank of England.
    16. David Laidler, 2010. "Chicago Monetary Traditions," Chapters, in: Ross B. Emmett (ed.), The Elgar Companion to the Chicago School of Economics, chapter 6, Edward Elgar Publishing.
    17. Willem H. Buiter, 2003. "James Tobin: An Appreciation of his Contribution to Economics," Economic Journal, Royal Economic Society, vol. 113(491), pages 585-631, November.
    18. John Duffy, 2022. "Why macroeconomics needs experimental evidence," The Japanese Economic Review, Springer, vol. 73(1), pages 5-29, January.
    19. Salle, Isabelle & Yıldızoğlu, Murat & Sénégas, Marc-Alexandre, 2013. "Inflation targeting in a learning economy: An ABM perspective," Economic Modelling, Elsevier, vol. 34(C), pages 114-128.
    20. Galbács, Péter, 2019. "A chicagonomics és a közgazdaságtan imperializmusa ["Chicagonomics" and the imperialism of economics]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(3), pages 229-255.

    More about this item

    Keywords

    expectation formation; exploration; deep reinforcement learning; bounded rationality; stochastic optimal growth;
    All these keywords.

    JEL classification:

    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • D84 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Expectations; Speculations
    • E21 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Consumption; Saving; Wealth
    • E70 - Macroeconomics and Monetary Economics - - Macro-Based Behavioral Economics - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_9255. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.