IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/25217.html
   My bibliography  Save this paper

On the Informativeness of Descriptive Statistics for Structural Estimates

Author

Listed:
  • Isaiah Andrews
  • Matthew Gentzkow
  • Jesse M. Shapiro

Abstract

We propose a way to formalize the relationship between descriptive analysis and structural estimation. A researcher reports an estimate ĉ of a structural quantity of interest c that is exactly or asymptotically unbiased under some base model. The researcher also reports descriptive statistics γ̂ that estimate features γ of the distribution of the data that are related to c under the base model. A reader entertains a less restrictive model that is local to the base model, under which the estimate ĉ may be biased. We study the reduction in worst-case bias from a restriction that requires the reader's model to respect the relationship between c and γ specified by the base model. Our main result shows that the proportional reduction in worst-case bias depends only on a quantity we call the informativeness of γ̂ for ĉ . Informativeness can be easily estimated even for complex models. We recommend that researchers report estimated informativeness alongside their descriptive analyses, and we illustrate with applications to three recent papers.

Suggested Citation

  • Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2018. "On the Informativeness of Descriptive Statistics for Structural Estimates," NBER Working Papers 25217, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:25217
    Note: DEV IO PE TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w25217.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    2. Liran Einav & Amy Finkelstein & Stephen P. Ryan & Paul Schrimpf & Mark R. Cullen, 2013. "Selection on Moral Hazard in Health Insurance," American Economic Review, American Economic Association, vol. 103(1), pages 178-219, February.
    3. Ariel Pakes, 2014. "Behavioral and Descriptive Forms of Choice Models," NBER Working Papers 20022, National Bureau of Economic Research, Inc.
    4. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Misspecification," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 6, pages 155-216, World Scientific Publishing Co. Pte. Ltd..
    5. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    6. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    7. Hansen, Bruce E., 2016. "Efficient shrinkage in parametric models," Journal of Econometrics, Elsevier, vol. 190(1), pages 115-132.
    8. Stéphane Bonhomme & Martin Weidner, 2018. "Minimizing sensitivity to model misspecification," CeMMAP working papers CWP59/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    9. Lars P. Hansen & Thomas J. Sargent, 2016. "Sets of Models and Prices of Uncertainty," NBER Working Papers 22000, National Bureau of Economic Research, Inc.
    10. Hansen, Lars-Peter & Sargent, Thomas-J, 2001. "Acknowledgement Misspecification in Macroeconomic Theory," Monetary and Economic Studies, Institute for Monetary and Economic Studies, Bank of Japan, vol. 19(S1), pages 213-227, February.
    11. Hidehiko Ichimura & Whitney K. Newey, 2022. "The influence function of semiparametric estimators," Quantitative Economics, Econometric Society, vol. 13(1), pages 29-61, January.
    12. Matthew Gentzkow, 2007. "Valuing New Goods in a Model with Complementarity: Online Newspapers," American Economic Review, American Economic Association, vol. 97(3), pages 713-744, June.
    13. Vivi Alatas & Abhijit Banerjee & Rema Hanna & Benjamin A. Olken & Ririn Purnamasari & Matthew Wai-Poi, 2016. "Self-Targeting: Evidence from a Field Experiment in Indonesia," Journal of Political Economy, University of Chicago Press, vol. 124(2), pages 371-427.
    14. Daniel K. Fetter & Lee M. Lockwood, 2018. "Government Old-Age Support and Labor Supply: Evidence from the Old Age Assistance Program," American Economic Review, American Economic Association, vol. 108(8), pages 2174-2211, August.
    15. Berkowitz, Daniel & Caner, Mehmet & Fang, Ying, 2008. "Are "Nearly Exogenous Instruments" reliable?," Economics Letters, Elsevier, vol. 101(1), pages 20-23, October.
    16. Orazio P. Attanasio & Costas Meghir & Ana Santiago, 2012. "Education Choices in Mexico: Using a Structural Model and a Randomized Experiment to Evaluate PROGRESA," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(1), pages 37-66.
    17. Emi Nakamura & Jón Steinsson, 2018. "Identification in Macroeconomics," Journal of Economic Perspectives, American Economic Association, vol. 32(3), pages 59-86, Summer.
    18. Nathaniel Hendren, 2013. "Private Information and Insurance Rejections," Econometrica, Econometric Society, vol. 81(5), pages 1713-1762, September.
    19. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    20. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    21. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    22. Matthew Gentzkow & Jesse M. Shapiro & Michael Sinkinson, 2014. "Competition and Ideological Diversity: Historical Evidence from US Newspapers," American Economic Review, American Economic Association, vol. 104(10), pages 3073-3114, October.
    23. Esther Duflo & Rema Hanna & Stephen P. Ryan, 2012. "Incentives Work: Getting Teachers to Come to School," American Economic Review, American Economic Association, vol. 102(4), pages 1241-1278, June.
    24. Jörg L. Spenkuch & B. Pablo Montagnes & Daniel B. Magleby, 2018. "Backward Induction in the Wild? Evidence from Sequential Voting in the US Senate," American Economic Review, American Economic Association, vol. 108(7), pages 1971-2013, July.
    25. Stéphane Bonhomme & Martin Weidner, 2020. "Minimizing Sensitivity to Model Misspecification," CeMMAP working papers CWP37/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    26. Guggenberger, Patrik, 2012. "On The Asymptotic Size Distortion Of Tests When Instruments Locally Violate The Exogeneity Assumption," Econometric Theory, Cambridge University Press, vol. 28(2), pages 387-421, April.
    27. K. Newey, Whitney, 1985. "Generalized method of moments specification testing," Journal of Econometrics, Elsevier, vol. 29(3), pages 229-256, September.
    28. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2017. "Measuring the Sensitivity of Parameter Estimates to Estimation Moments," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(4), pages 1553-1592.
    29. Dridi, Ramdan & Guay, Alain & Renault, Eric, 2007. "Indirect inference and calibration of dynamic stochastic general equilibrium models," Journal of Econometrics, Elsevier, vol. 136(2), pages 397-430, February.
    30. Yaroslav Mukhin, 2018. "Sensitivity of Regular Estimators," Papers 1805.08883, arXiv.org.
    31. Melanie Morten, 2019. "Temporary Migration and Endogenous Risk Sharing in Village India," Journal of Political Economy, University of Chicago Press, vol. 127(1), pages 1-46.
    32. Yuichi Kitamura & Taisuke Otsu & Kirill Evdokimov, 2013. "Robustness, Infinitesimal Neighborhoods, and Moment Restrictions," Econometrica, Econometric Society, vol. 81(3), pages 1185-1201, May.
    33. Hansen, Lars Peter & Sargent, Thomas J., 2005. "Robust estimation and control under commitment," Journal of Economic Theory, Elsevier, vol. 124(2), pages 258-301, October.
    34. Matzkin, Rosa L., 2007. "Nonparametric identification," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 73, Elsevier.
    35. Lars Peter Hansen & Thomas J. Sargent, 2001. "Acknowledging Misspecification in Macroeconomic Theory," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 4(3), pages 519-535, July.
    36. Ariel Pakes, 2014. "The 2013 Lawrence R. Klein Lecture: Behavioral And Descriptive Forms Of Choice Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 55, pages 603-624, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    2. Philipp Eisenhauer & Lena Janys & Christopher Walsh & Janós Gabler, 2023. "Structural Models for Policy-Making," CRC TR 224 Discussion Paper Series crctr224_2023_484, University of Bonn and University of Mannheim, Germany.
    3. Stéphane Bonhomme & Martin Weidner, 2018. "Minimizing sensitivity to model misspecification," CeMMAP working papers CWP59/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    4. Maximilian Blesch & Philipp Eisenhauer, 2021. "Robust decision-making under risk and ambiguity," Papers 2104.12573, arXiv.org, revised Oct 2021.
    5. Stéphane Bonhomme & Martin Weidner, 2020. "Minimizing Sensitivity to Model Misspecification," CeMMAP working papers CWP37/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. St'ephane Bonhomme & Martin Weidner, 2019. "Posterior Average Effects," Papers 1906.06360, arXiv.org, revised Sep 2021.
    7. Nathan Canen & Kyungchul Song, 2021. "Counterfactual analysis under partial identification using locally robust refinement," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(4), pages 416-436, June.
    8. Stéphane Bonhomme & Martin Weidner, 2022. "Minimizing sensitivity to model misspecification," Quantitative Economics, Econometric Society, vol. 13(3), pages 907-954, July.
    9. Raffaella Giacomini & Toru Kitagawa & Matthew Read, 2021. "Robust Bayesian Analysis for Econometrics," Working Paper Series WP-2021-11, Federal Reserve Bank of Chicago.
    10. Maximilian Blesch & Philipp Eisenhauer, 2023. "Robust Decision-Making under Risk and Ambiguity," Rationality and Competition Discussion Paper Series 463, CRC TRR 190 Rationality and Competition.
    11. Arthur Lewbel, 2019. "The Identification Zoo: Meanings of Identification in Econometrics," Journal of Economic Literature, American Economic Association, vol. 57(4), pages 835-903, December.
    12. Philipp Eisenhauer & Janos Gabler & Lena Janys, 2021. "Structural Models for Policy-Making: Coping with Parametric Uncertainty," ECONtribute Discussion Papers Series 082, University of Bonn and University of Cologne, Germany.
    13. Eisenhauer, Philipp & Gabler, Janos & Janys, Lena, 2021. "Structural Models for Policy-Making: Coping with Parametric Uncertainty," IZA Discussion Papers 14317, Institute of Labor Economics (IZA).
    14. Jens Klooster & Mikhail Zhelonkin, 2024. "Outlier robust inference in the instrumental variable model with applications to causal effects," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(1), pages 86-106, January.
    15. Philipp Eisenhauer & Jano's Gabler & Lena Janys & Christopher Walsh, 2021. "Structural models for policy-making: Coping with parametric uncertainty," Papers 2103.01115, arXiv.org, revised Jun 2022.
    16. Walter Beckert & Daniel Kaliski, 2019. "Honest inference for discrete outcomes," CeMMAP working papers CWP67/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    17. Maximilian Blesch & Philipp Eisenhauer, 2021. "Robust Decision-Making Under Risk and Ambiguity," ECONtribute Discussion Papers Series 104, University of Bonn and University of Cologne, Germany.
    18. Timothy Christensen & Benjamin Connault, 2023. "Counterfactual Sensitivity and Robustness," Econometrica, Econometric Society, vol. 91(1), pages 263-298, January.
    19. Andres Santos, 2020. "A Comment on: “On the Informativeness of Descriptive Statistics for Structural Estimates” by Isaiah Andrews, Matthew Gentzkow, and Jesse M. Shapiro," Econometrica, Econometric Society, vol. 88(6), pages 2271-2276, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2020. "Transparency in Structural Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 711-722, October.
    2. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2017. "Measuring the Sensitivity of Parameter Estimates to Estimation Moments," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(4), pages 1553-1592.
    3. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    4. Stéphane Bonhomme & Martin Weidner, 2020. "Minimizing Sensitivity to Model Misspecification," CeMMAP working papers CWP37/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    5. Stéphane Bonhomme & Martin Weidner, 2022. "Minimizing sensitivity to model misspecification," Quantitative Economics, Econometric Society, vol. 13(3), pages 907-954, July.
    6. St'ephane Bonhomme & Martin Weidner, 2018. "Minimizing Sensitivity to Model Misspecification," Papers 1807.02161, arXiv.org, revised Oct 2021.
    7. Hwang, Jungbin & Kang, Byunghoon & Lee, Seojeong, 2022. "A doubly corrected robust variance estimator for linear GMM," Journal of Econometrics, Elsevier, vol. 229(2), pages 276-298.
    8. Byunghoon Kang, 2018. "Higher Order Approximation of IV Estimators with Invalid Instruments," Working Papers 257105320, Lancaster University Management School, Economics Department.
    9. Maibom, Jonas, 2021. "The Danish Labor Market Experiments: Methods and Findings," Nationaløkonomisk tidsskrift, Nationaløkonomisk Forening, vol. 2021(1), pages 1-21.
    10. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    11. Ruoyao Shi, 2021. "An Averaging Estimator for Two Step M Estimation in Semiparametric Models," Working Papers 202105, University of California at Riverside, Department of Economics.
    12. Naoya Sueishi, 2022. "A Misuse of Specification Tests," Papers 2211.11915, arXiv.org.
    13. Raffaella Giacomini & Toru Kitagawa & Harald Uhlig, 2019. "Estimation Under Ambiguity," CeMMAP working papers CWP24/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    14. Jiaming Mao & Zhesheng Zheng, 2020. "Structural Regularization," Papers 2004.12601, arXiv.org, revised Jun 2020.
    15. Claudia Noack, 2021. "Sensitivity of LATE Estimates to Violations of the Monotonicity Assumption," Papers 2106.06421, arXiv.org.
    16. Victor Chernozhukov & Juan Carlos Escanciano & Hidehiko Ichimura & Whitney K. Newey & James M. Robins, 2022. "Locally Robust Semiparametric Estimation," Econometrica, Econometric Society, vol. 90(4), pages 1501-1535, July.
    17. Matthew A. Masten & Alexandre Poirier, 2021. "Salvaging Falsified Instrumental Variable Models," Econometrica, Econometric Society, vol. 89(3), pages 1449-1469, May.
    18. Li, Jing, 2018. "Essays on model uncertainty in financial models," Other publications TiSEM 202cd910-7ef1-4db4-94ae-d, Tilburg University, School of Economics and Management.
    19. Timothy Christensen & Benjamin Connault, 2023. "Counterfactual Sensitivity and Robustness," Econometrica, Econometric Society, vol. 91(1), pages 263-298, January.
    20. Doko Tchatoka, Firmin Sabro, 2012. "Specification Tests with Weak and Invalid Instruments," MPRA Paper 40185, University Library of Munich, Germany.

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • D12 - Microeconomics - - Household Behavior - - - Consumer Economics: Empirical Analysis
    • I13 - Health, Education, and Welfare - - Health - - - Health Insurance, Public and Private
    • I25 - Health, Education, and Welfare - - Education - - - Education and Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25217. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.