IDEAS home Printed from https://ideas.repec.org/p/red/sed006/178.html
   My bibliography  Save this paper

Learning and Model Validation

Author

Listed:
  • In-Koo Cho

    (Economics University of Illinois)

  • Kenneth Kasa

Abstract

This paper studies the following problem. An agent takes actions based on a possibly misspecified model. The agent is 'large', in the sense that his actions influence the model he is trying to learn about. The agent is aware of potential model misspecification and tries to detect it, in real-time, using an econometric specification test. If his model fails the test, he formulates a new better-fitting model. If his model passes the test, he uses it to formulate and implement a policy based on the provisional assumption that the current model is correctly specified, and will not change in the future. We claim that this testing and model validation process is an accurate description of most macroeconomic policy problems. Unfortunately, the dynamics produced by this process are not well understood. We make progress on this problem by relating it to a problem that is well understood. In particular, we relate it to the dynamics of constant-gain stochastic approximation algorithms. This enables us to appeal to well known results from the large deviations literature to help us understand the dynamics of testing and model revision. We show that as the agent applies an increasingly stringent specification test, the large deviation properties of the discrete model validation dynamics converge to those of the continuous learning dynamics. This sheds new light on the recent constant-gain learning literature.

Suggested Citation

  • In-Koo Cho & Kenneth Kasa, 2006. "Learning and Model Validation," 2006 Meeting Papers 178, Society for Economic Dynamics.
  • Handle: RePEc:red:sed006:178
    as

    Download full text from publisher

    File URL: http://www.sfu.ca/~kkasa/validate.PDF
    File Function: main text
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. William Branch & George W. Evans, 2007. "Model Uncertainty and Endogenous Volatility," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 10(2), pages 207-237, April.
    2. I. Gilboa & W. A. Postlewaite & D. Schmeidler, 2009. "Probability and Uncertainty in Economic Modeling," Voprosy Ekonomiki, NP Voprosy Ekonomiki, issue 10.
    3. Timothy Cogley & Riccardo Colacito & Thomas J. Sargent, 2007. "Benefits from U.S. Monetary Policy Experimentation in the Days of Samuelson and Solow and Lucas," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 67-99, February.
    4. Timothy Cogley & Thomas J. Sargent, 2008. "Anticipated Utility And Rational Expectations As Approximations Of Bayesian Decision Making," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 49(1), pages 185-221, February.
    5. Sin, Chor-Yiu & White, Halbert, 1996. "Information criteria for selecting possibly misspecified parametric models," Journal of Econometrics, Elsevier, vol. 71(1-2), pages 207-225.
    6. Marcet, Albert & Sargent, Thomas J., 1989. "Convergence of least squares learning mechanisms in self-referential linear stochastic models," Journal of Economic Theory, Elsevier, vol. 48(2), pages 337-368, August.
    7. Inoue, Atsushi & Rossi, Barbara, 2005. "Recursive Predictability Tests for Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 336-345, July.
    8. Foster, Dean P. & Young, H. Peyton, 2003. "Learning, hypothesis testing, and Nash equilibrium," Games and Economic Behavior, Elsevier, vol. 45(1), pages 73-96, October.
    9. Albert Marcet & Juan P. Nicolini, 2003. "Recurrent Hyperinflations and Learning," American Economic Review, American Economic Association, vol. 93(5), pages 1476-1498, December.
    10. Kalai, Ehud & Lehrer, Ehud, 1993. "Rational Learning Leads to Nash Equilibrium," Econometrica, Econometric Society, vol. 61(5), pages 1019-1045, September.
    11. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    12. Bray, Margaret, 1982. "Learning, estimation, and the stability of rational expectations," Journal of Economic Theory, Elsevier, vol. 26(2), pages 318-339, April.
    13. In-Koo Cho & Noah Williams & Thomas J. Sargent, 2002. "Escaping Nash Inflation," Review of Economic Studies, Oxford University Press, vol. 69(1), pages 1-40.
    14. Brock, William A. & Durlauf, Steven N. & West, Kenneth D., 2007. "Model uncertainty and policy evaluation: Some theory and empirics," Journal of Econometrics, Elsevier, vol. 136(2), pages 629-664, February.
    15. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
    16. Aghion, Philippe & Bacchetta, Philippe & Banerjee, Abhijit, 2001. "Currency crises and monetary policy in an economy with credit constraints," European Economic Review, Elsevier, vol. 45(7), pages 1121-1150.
    17. Yuichi Kitamura & Michael Stutzer, 1997. "An Information-Theoretic Alternative to Generalized Method of Moments Estimation," Econometrica, Econometric Society, vol. 65(4), pages 861-874, July.
    18. Kenneth Kasa, 2004. "Learning, Large Deviations, And Recurrent Currency Crises," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(1), pages 141-173, February.
    19. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    20. Thomas J. Sargent & Noah Williams, 2005. "Impacts of Priors on Convergence and Escapes from Nash Inflation," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 8(2), pages 360-391, April.
    21. repec:hrv:faseco:4729511 is not listed on IDEAS
    22. Sargent, Thomas J, 1976. "The Observational Equivalence of Natural and Unnatural Rate Theories of Macroeconomics," Journal of Political Economy, University of Chicago Press, vol. 84(3), pages 631-640, June.
    23. Timothy Cogley & Thomas J. Sargent, 2005. "Drift and Volatilities: Monetary Policies and Outcomes in the Post WWII U.S," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 8(2), pages 262-302, April.
    24. Sungbae An & Frank Schorfheide, 2007. "Bayesian Analysis of DSGE Models," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 113-172.
    25. Brunner, Karl & Meltzer, Allan H., 1976. "The Phillips curve," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 1-18, January.
    26. McGough, Bruce, 2003. "Statistical Learning With Time-Varying Parameters," Macroeconomic Dynamics, Cambridge University Press, vol. 7(1), pages 119-139, February.
    27. William A. Brock & Cars H. Hommes, 1997. "A Rational Route to Randomness," Econometrica, Econometric Society, vol. 65(5), pages 1059-1096, September.
    28. Chen, Xiaohong & White, Halbert, 1998. "Nonparametric Adaptive Learning with Feedback," Journal of Economic Theory, Elsevier, vol. 82(1), pages 190-222, September.
    29. Kandori, Michihiro & Mailath, George J & Rob, Rafael, 1993. "Learning, Mutation, and Long Run Equilibria in Games," Econometrica, Econometric Society, vol. 61(1), pages 29-56, January.
    30. Timothy Cogley & Thomas Sargent & Riccardo Colacito, 2005. "Benefits from U.S. Monetary Policy Experimentation in the Days of Samuelson," 2005 Meeting Papers 791, Society for Economic Dynamics.
    31. Sargent, Thomas J., 1993. "Bounded Rationality in Macroeconomics: The Arne Ryde Memorial Lectures," OUP Catalogue, Oxford University Press, number 9780198288695.
    32. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    33. Lars Peter Hansen & Thomas J Sargent, 2014. "A Quartet of Semigroups for Model Specification, Robustness, Prices of Risk, and Model Detection," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 4, pages 83-143, World Scientific Publishing Co. Pte. Ltd..
    34. Margaret Bray & David M. Kreps, 1987. "Rational Learning and Rational Expectations," Palgrave Macmillan Books, in: George R. Feiwel (ed.), Arrow and the Ascent of Modern Economic Theory, chapter 19, pages 597-625, Palgrave Macmillan.
    35. Hansen, Lars Peter & Sargent, Thomas J., 1993. "Seasonality and approximation errors in rational expectations models," Journal of Econometrics, Elsevier, vol. 55(1-2), pages 21-55.
    36. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    37. Timothy Cogley & Thomas J. Sargent, 2008. "Anticipated Utility And Rational Expectations As Approximations Of Bayesian Decision Making," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 49(1), pages 185-221, February.
    38. Christopher A. Sims, 1982. "Policy Analysis with Econometric Models," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 13(1), pages 107-164.
    39. Shen X. & Ye J., 2002. "Adaptive Model Selection," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 210-221, March.
    40. Chu, Chia-Shang James & Stinchcombe, Maxwell & White, Halbert, 1996. "Monitoring Structural Change," Econometrica, Econometric Society, vol. 64(5), pages 1045-1065, September.
    41. Bullard, James, 1992. "Time-varying parameters and nonconvergence to rational expectations under least squares learning," Economics Letters, Elsevier, vol. 40(2), pages 159-166, October.
    42. Massimiliano Marcellino, "undated". "Model Selection for Non-Linear Dynamic Models," Working Papers 159, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    43. Douglas Rivers & Quang Vuong, 2002. "Model selection tests for nonlinear dynamic models," Econometrics Journal, Royal Economic Society, vol. 5(1), pages 1-39, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. In-Koo Cho & Ken Kasa, 2012. "Model Validation and Learning," Discussion Papers dp12-07, Department of Economics, Simon Fraser University.
    2. In-Koo Cho & Kenneth Kasa, 2017. "Gresham's Law of Model Averaging," American Economic Review, American Economic Association, vol. 107(11), pages 3589-3616, November.
    3. Norman, Thomas W.L., 2015. "Learning, hypothesis testing, and rational-expectations equilibrium," Games and Economic Behavior, Elsevier, vol. 90(C), pages 93-105.
    4. Berardi, Michele & Galimberti, Jaqueson K., 2017. "Empirical calibration of adaptive learning," Journal of Economic Behavior & Organization, Elsevier, vol. 144(C), pages 219-237.
    5. Berardi, Michele & Galimberti, Jaqueson K., 2013. "A note on exact correspondences between adaptive learning algorithms and the Kalman filter," Economics Letters, Elsevier, vol. 118(1), pages 139-142.
    6. repec:zbw:bofrdp:2007_032 is not listed on IDEAS
    7. Dmitri Kolyuzhnov & Anna Bogomolova, 2004. "Escape Dynamics: A Continuous Time Approximation," Econometric Society 2004 Latin American Meetings 27, Econometric Society.
    8. Guidolin, Massimo & Timmermann, Allan, 2007. "Properties of equilibrium asset prices under alternative learning schemes," Journal of Economic Dynamics and Control, Elsevier, vol. 31(1), pages 161-217, January.
    9. William Branch & George W. Evans, 2007. "Model Uncertainty and Endogenous Volatility," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 10(2), pages 207-237, April.
    10. Evans, David & Evans, George W. & McGough, Bruce, 2022. "The RPEs of RBCs and other DSGEs," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    11. Kolyuzhnov, Dmitri & Bogomolova, Anna & Slobodyan, Sergey, 2014. "Escape dynamics: A continuous-time approximation," Journal of Economic Dynamics and Control, Elsevier, vol. 38(C), pages 161-183.
    12. George W. Evans & Seppo Honkapohja, 2009. "Expectations, Learning and Monetary Policy: An Overview of Recent Research," Central Banking, Analysis, and Economic Policies Book Series, in: Klaus Schmidt-Hebbel & Carl E. Walsh & Norman Loayza (Series Editor) & Klaus Schmidt-Hebbel (Series (ed.),Monetary Policy under Uncertainty and Learning, edition 1, volume 13, chapter 2, pages 027-076, Central Bank of Chile.
    13. Consolo, Agostino & Favero, Carlo A. & Paccagnini, Alessia, 2009. "On the statistical identification of DSGE models," Journal of Econometrics, Elsevier, vol. 150(1), pages 99-115, May.
    14. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," Review of Economic Studies, Oxford University Press, vol. 74(4), pages 1059-1087.
    15. Dmitri Kolyuzhnov & Anna Bogomolova, 2004. "Escape Dynamics: A Continuous Time Approximation," Econometric Society 2004 Far Eastern Meetings 557, Econometric Society.
    16. Francesca Rondina, 2017. "Model Uncertainty and the Direction of Fit of the Postwar U.S. Phillips Curve(s)," Working Papers 1702E, University of Ottawa, Department of Economics.
    17. Dmitri Kolyuzhnov & Anna Bogomolova, 2004. "Escape Dynamics: A Continuous Time Approximation," Computing in Economics and Finance 2004 190, Society for Computational Economics.
    18. George W. Evans & Seppo Honkapohja, 2009. "Expectations, Learning and Monetary Policy: An Overview of Recent Research," Central Banking, Analysis, and Economic Policies Book Series, in: Klaus Schmidt-Hebbel & Carl E. Walsh & Norman Loayza (Series Editor) & Klaus Schmidt-Hebbel (Series (ed.),Monetary Policy under Uncertainty and Learning, edition 1, volume 13, chapter 2, pages 027-076, Central Bank of Chile.
    19. Ellison, Martin & Scott, Andrew, 2013. "Learning and price volatility in duopoly models of resource depletion," Journal of Monetary Economics, Elsevier, vol. 60(7), pages 806-820.
    20. Felipe Pérez, 1998. "- Private Experience In Adaptive Learning Models," Working Papers. Serie AD 1998-03, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    21. White, Halbert & Pettenuzzo, Davide, 2014. "Granger causality, exogeneity, cointegration, and economic policy analysis," Journal of Econometrics, Elsevier, vol. 178(P2), pages 316-330.

    More about this item

    Keywords

    Learning; Validation; Relative Entropy; Large Deviation;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • E59 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - Other

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:red:sed006:178. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christian Zimmermann (email available below). General contact details of provider: https://edirc.repec.org/data/sedddea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.