IDEAS home Printed from https://ideas.repec.org/a/aea/jeclit/v52y2014i3p820-50.html
   My bibliography  Save this article

The Limits of Inference with Theory: A Review of Wolpin (2013)

Author

Listed:
  • John Rust

Abstract

This essay reviews Kenneth I. Wolpin's (2013) monograph The Limits of Inference without Theory, which arose from lectures he presented at the Cowles Foundation in 2010 in honor of Tjalling Koopmans. While I readily agree with Wolpin's basic premise that empirical work that eschews the role of economic theory faces unnecessary self-imposed limits relative to empirical work that embraces and tries to test and improve economic theory, it is important to be aware that the use of economic theory is not a panacea. I point out that there are also serious limits to inference with theory: 1) there may be no truly "structural" (policy invariant) parameters, a key assumption underpinning the structural econometric approach that Wolpin and the Cowles Foundation have championed; 2) there is a curse of dimensionality that makes it very difficult for us to elucidate the detailed implications of economic theories, which is necessary to empirically implement and test these theories; 3) there is an identification problem that makes it impossible to decide between competing theories without imposing ad hoc auxiliary assumptions (such as parametric functional form assumptions); and 4) there is a problem of multiplicity and indeterminacy of equilibria that limits the predictive empirical content of many economic theories. I conclude that though these are very challenging problems, I agree with Wolpin and the Cowles Foundation that economists have far more to gain by trying to incorporate economic theory into empirical work and test and improve our theories than by rejecting theory and presuming that all interesting economic issues can be answered by well-designed controlled, randomized experiments and assuming that difficult questions of causality and evaluation of alternative hypothetical policies can be resolved by simply allowing the "data to speak for itself."

Suggested Citation

  • John Rust, 2014. "The Limits of Inference with Theory: A Review of Wolpin (2013)," Journal of Economic Literature, American Economic Association, vol. 52(3), pages 820-850, September.
  • Handle: RePEc:aea:jeclit:v:52:y:2014:i:3:p:820-50
    Note: DOI: 10.1257/jel.52.3.820
    as

    Download full text from publisher

    File URL: http://www.aeaweb.org/articles.php?doi=10.1257/jel.52.3.820
    Download Restriction: no

    File URL: http://www.aeaweb.org/jel/ds/5203/JEL.52.3.375_ds.zip
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    References listed on IDEAS

    as
    1. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    2. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    3. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    4. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    5. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    6. Michael P. Keane & Kenneth I. Wolpin, 2010. "The Role Of Labor And Marriage Markets, Preference Heterogeneity, And The Welfare System In The Life Cycle Decisions Of Black, Hispanic, And White Women," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 51(3), pages 851-892, August.
    7. Victor Aguirregabiria & Pedro Mira, 2013. "Identification of Games of Incomplete Information with Multiple Equilibria and Common Unobserved Heterogeneity," Working Papers tecipa-474, University of Toronto, Department of Economics.
    8. Sanjog Misra & Harikesh Nair, 2011. "A structural model of sales-force compensation dynamics: Estimation and field implementation," Quantitative Marketing and Economics (QME), Springer, vol. 9(3), pages 211-257, September.
    9. Gilboa, Itzhak & Samuelson, Larry & Schmeidler, David, 2013. "Dynamics of inductive inference in a unified framework," Journal of Economic Theory, Elsevier, vol. 148(4), pages 1399-1432.
    10. Matzkin, Rosa L, 1991. "Semiparametric Estimation of Monotone and Concave Utility Functions for Polychotomous Choice Models," Econometrica, Econometric Society, vol. 59(5), pages 1315-1327, September.
    11. Thierry Magnac & David Thesmar, 2002. "Identifying Dynamic Discrete Decision Processes," Econometrica, Econometric Society, vol. 70(2), pages 801-816, March.
    12. Heckman, James J. & Navarro, Salvador, 2007. "Dynamic discrete choice and dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 136(2), pages 341-396, February.
    13. repec:pri:rpdevs:instruments_of_development is not listed on IDEAS
    14. Fedor Iskhakov & John Rust & Bertel Schjerning, 2016. "Recursive Lexicographical Search: Finding All Markov Perfect Equilibria of Finite State Directional Dynamic Games," Review of Economic Studies, Oxford University Press, vol. 83(2), pages 658-703.
    15. J. Rust & J. F. Traub & H. Wozniakowski, 2002. "Is There a Curse of Dimensionality for Contraction Fixed Points in the Worst Case?," Econometrica, Econometric Society, vol. 70(1), pages 285-329, January.
    16. John Rust & Christopher Phelan, 1997. "How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets," Econometrica, Econometric Society, vol. 65(4), pages 781-832, July.
    17. Kenneth I. Wolpin & Petra E. Todd, 2006. "Assessing the Impact of a School Subsidy Program in Mexico: Using a Social Experiment to Validate a Dynamic Behavioral Model of Child Schooling and Fertility," American Economic Review, American Economic Association, vol. 96(5), pages 1384-1417, December.
    18. Wolpin, Kenneth I., 2013. "The Limits of Inference without Theory," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262019086, December.
    19. Heckman, James J, 1992. "Haavelmo and the Birth of Modern Econometrics: A Review of The History of Econometric Ideas," Journal of Economic Literature, American Economic Association, vol. 30(2), pages 876-886, June.
    20. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    21. D. COLANDER & al., 2010. "The Financial Crisis and the Systemic Failure of Academic Economics," VOPROSY ECONOMIKI, N.P. Redaktsiya zhurnala "Voprosy Economiki", vol. 6.
    22. John Geweke, 2014. "Review Essay on Charles F. Manski's Public Policy in an Uncertain World: Analysis and Decisions," Journal of Economic Literature, American Economic Association, vol. 52(3), pages 799-804, September.
    23. Gilboa, Itzhak & Schmeidler, David, 1989. "Maxmin expected utility with non-unique prior," Journal of Mathematical Economics, Elsevier, vol. 18(2), pages 141-153, April.
    24. Peter C. B. Phillips, 2003. "Laws and Limits of Econometrics," Economic Journal, Royal Economic Society, vol. 113(486), pages 26-52, March.
    25. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    26. Aviv Nevo & Michael D. Whinston, 2010. "Taking the Dogma out of Econometrics: Structural Modeling and Credible Inference," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 69-82, Spring.
    27. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    28. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    29. Schorfheide, Frank & Wolpin, Kenneth I., 2016. "To hold out or not to hold out," Research in Economics, Elsevier, vol. 70(2), pages 332-345.
    30. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    31. Morgan,Mary S., 1991. "The History of Econometric Ideas," Cambridge Books, Cambridge University Press, number 9780521424653.
    32. Manski, Charles F., 2013. "Public Policy in an Uncertain World: Analysis and Decisions," Economics Books, Harvard University Press, number 9780674066892, Spring.
    33. John Rust, 1997. "Using Randomization to Break the Curse of Dimensionality," Econometrica, Econometric Society, vol. 65(3), pages 487-516, May.
    34. Einhorn, Hillel J & Hogarth, Robin M, 1986. "Decision Making under Ambiguity," The Journal of Business, University of Chicago Press, vol. 59(4), pages 225-250, October.
    35. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    36. Hanming Fang & Alessandro Gavazza, 2011. "Dynamic Inefficiencies in an Employment-Based Health Insurance System: Theory and Evidence," American Economic Review, American Economic Association, vol. 101(7), pages 3047-3077, December.
    37. Joseph E. Stiglitz, 2002. "Information and the Change in the Paradigm in Economics," American Economic Review, American Economic Association, vol. 92(3), pages 460-501, June.
    38. Paul Frijters, 2013. "The Limits of Inference Without Theory , by Kenneth I. Wolpin ( MIT University Press , Cambridge, MA , 2013 ), pp. 192 ," The Economic Record, The Economic Society of Australia, vol. 89(286), pages 429-432, September.
    39. Rust, John, 2010. "Comments on: "Structural vs. atheoretic approaches to econometrics" by Michael Keane," Journal of Econometrics, Elsevier, vol. 156(1), pages 21-24, May.
    40. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yew-Kwang Ng, 2016. "Are Unrealistic Assumptions/Simplifications Acceptable? Some Methodological Issues in Economics," Pacific Economic Review, Wiley Blackwell, vol. 21(2), pages 180-201, May.
    2. Jiaming Mao & Zhesheng Zheng, 2020. "Structural Regularization," Papers 2004.12601, arXiv.org, revised Jun 2020.
    3. Linli Xu & Jorge M. Silva-Risso & Kenneth C. Wilbur, 2018. "Dynamic Quality Ladder Model Predictions in Nonrandom Holdout Samples," Management Science, INFORMS, vol. 64(7), pages 3187-3207, July.
    4. Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
    5. Jiaming Mao & Jingzhi Xu, 2020. "Ensemble Learning with Statistical and Structural Models," Papers 2006.05308, arXiv.org.
    6. Drerup, Tilman & Enke, Benjamin & von Gaudecker, Hans-Martin, 2014. "Measurement Error in Subjective Expectation and the Empirical Content of Economic Models," MEA discussion paper series 201414, Munich Center for the Economics of Aging (MEA) at the Max Planck Institute for Social Law and Social Policy.
    7. Myrto Kalouptsidi & Paul T. Scott & Eduardo Souza-Rodrigues, 2015. "Identification of Counterfactuals in Dynamic Discrete Choice Models," NBER Working Papers 21527, National Bureau of Economic Research, Inc.
    8. Breitmoser, Yves, 2017. "Discrete Choice with Presentation Effects," Rationality and Competition Discussion Paper Series 35, CRC TRR 190 Rationality and Competition.
    9. Myrto Kalouptsidi & Paul T. Scott & Eduardo Souza-Rodrigues, 2015. "Identification of Counterfactuals and Payoffs in Dynamic Discrete Choice with an Application to Land Use," Working Papers tecipa-546, University of Toronto, Department of Economics.
    10. Gilles Chemla & Christopher Hennessy, 2021. "Equilibrium Counterfactuals," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 62(2), pages 639-669, May.
    11. Breitmoser, Yves, 2018. "The Axiomatic Foundation of Logit," Rationality and Competition Discussion Paper Series 78, CRC TRR 190 Rationality and Competition.
    12. Akhabbar, Amanar, 2019. "Introduction : Malaise dans la science économique ? [Introduction: Economics and Its Discontents]," MPRA Paper 93328, University Library of Munich, Germany.
    13. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    14. Hansen, Jörgen & Kristensen, Nicolai & Andersen, Henrik Lindegaard, 2020. "The Bottom 20%: Early Career Paths of Adolescents with Low GPA," IZA Discussion Papers 13564, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    2. Dagsvik, John K., 2018. "Invariance axioms and functional form restrictions in structural models," Mathematical Social Sciences, Elsevier, vol. 91(C), pages 85-95.
    3. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    4. Dagsvik, John K, 2017. "Invariance Axioms and Functional Form Restrictions in Structural Models," Memorandum 08/2017, Oslo University, Department of Economics.
    5. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    6. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    7. Thoresen, Thor O. & Vattø, Trine E., 2015. "Validation of the discrete choice labor supply model by methods of the new tax responsiveness literature," Labour Economics, Elsevier, vol. 37(C), pages 38-53.
    8. Maibom, Jonas, 2021. "The Danish Labor Market Experiments: Methods and Findings," Nationaløkonomisk tidsskrift, Nationaløkonomisk Forening, vol. 2021(1), pages 1-21.
    9. Lechner, Michael, 2013. "Treatment effects and panel data," Economics Working Paper Series 1314, University of St. Gallen, School of Economics and Political Science.
    10. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    11. Kalouptsidi, Myrto & Scott, Paul & Souza-Rodrigues, Eduardo, 2017. "Identification of Counterfactuals in Dynamic Discrete Choice Models," CEPR Discussion Papers 12470, C.E.P.R. Discussion Papers.
    12. Ziebarth, Nicolas R., 2017. "Social Insurance and Health," IZA Discussion Papers 10918, Institute of Labor Economics (IZA).
    13. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    14. Denni Tommasi & Arthur Lewbel & Rossella Calvi, 2017. "LATE with Mismeasured or Misspecified Treatment: An application to Women's Empowerment in India," Working Papers ECARES ECARES 2017-27, ULB -- Universite Libre de Bruxelles.
    15. Heckman, James J. & Humphries, John Eric & Veramendi, Gregory, 2016. "Dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 191(2), pages 276-292.
    16. Jiaming Mao & Jingzhi Xu, 2020. "Ensemble Learning with Statistical and Structural Models," Papers 2006.05308, arXiv.org.
    17. Rothstein, Jesse & von Wachter, Till, 2016. "Social Experiments in the Labor Market," Institute for Research on Labor and Employment, Working Paper Series qt6605k20b, Institute of Industrial Relations, UC Berkeley.
    18. Jiaming Mao & Zhesheng Zheng, 2020. "Structural Regularization," Papers 2004.12601, arXiv.org, revised Jun 2020.
    19. Duo Qin & Yanqun Zhang, 2013. "A History of Polyvalent Structural Parameters: the Case of Instrument Variable Estimators," Working Papers 183, Department of Economics, SOAS, University of London, UK.
    20. Dionissi Aliprantis, 2011. "Assessing the evidence on neighborhood effects from moving to opportunity," Working Papers (Old Series) 1101, Federal Reserve Bank of Cleveland.

    More about this item

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jeclit:v:52:y:2014:i:3:p:820-50. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.