IDEAS home Printed from https://ideas.repec.org/a/oup/qjecon/v132y2017i4p1553-1592..html
   My bibliography  Save this article

Measuring the Sensitivity of Parameter Estimates to Estimation Moments

Author

Listed:
  • Isaiah Andrews
  • Matthew Gentzkow
  • Jesse M. Shapiro

Abstract

We propose a local measure of the relationship between parameter estimates and the moments of the data they depend on. Our measure can be computed at negligible cost even for complex structural models. We argue that reporting this measure can increase the transparency of structural estimates, making it easier for readers to predict the way violations of identifying assumptions would affect the results. When the key assumptions are orthogonality between error terms and excluded instruments, we show that our measure provides a natural extension of the omitted variables bias formula for nonlinear models. We illustrate with applications to published articles in several fields of economics.

Suggested Citation

  • Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2017. "Measuring the Sensitivity of Parameter Estimates to Estimation Moments," The Quarterly Journal of Economics, Oxford University Press, vol. 132(4), pages 1553-1592.
  • Handle: RePEc:oup:qjecon:v:132:y:2017:i:4:p:1553-1592.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/qje/qjx023
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    2. Jean‐Pierre Dubé & Jeremy T. Fox & Che‐Lin Su, 2012. "Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation," Econometrica, Econometric Society, vol. 80(5), pages 2231-2267, September.
    3. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    4. Stefano DellaVigna & John A. List & Ulrike Malmendier, 2012. "Testing for Altruism and Social Pressure in Charitable Giving," The Quarterly Journal of Economics, Oxford University Press, vol. 127(1), pages 1-56.
    5. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    6. Liran Einav & Amy Finkelstein & Stephen P. Ryan & Paul Schrimpf & Mark R. Cullen, 2013. "Selection on Moral Hazard in Health Insurance," American Economic Review, American Economic Association, vol. 103(1), pages 178-219, February.
    7. Ariel Pakes, 2003. "Common Sense and Simplicity in Empirical Industrial Organization," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 23(3_4), pages 193-215, December.
    8. Kei Kawai & Yasutora Watanabe, 2013. "Inferring Strategic Voting," American Economic Review, American Economic Association, vol. 103(2), pages 624-662, April.
    9. Ronchetti, Elvezio & Trojani, Fabio, 2001. "Robust inference with GMM estimators," Journal of Econometrics, Elsevier, vol. 101(1), pages 37-69, March.
    10. Levon Barseghyan & Francesca Molinari & Ted O'Donoghue & Joshua C. Teitelbaum, 2013. "The Nature of Risk Preferences: Evidence from Insurance Choices," American Economic Review, American Economic Association, vol. 103(6), pages 2499-2529, October.
    11. Ying Fan, 2013. "Ownership Consolidation and Product Characteristics: A Study of the US Daily Newspaper Market," American Economic Review, American Economic Association, vol. 103(5), pages 1598-1628, August.
    12. Steven Berry & Alon Eizenberg & Joel Waldfogel, 2016. "Optimal product variety in radio markets," RAND Journal of Economics, RAND Corporation, vol. 47(3), pages 463-497, August.
    13. Aviv Nevo & Adam M. Rosen, 2012. "Identification With Imperfect Instruments," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 659-671, August.
    14. Aviv Nevo, 2000. "Mergers with Differentiated Products: The Case of the Ready-to-Eat Cereal Industry," RAND Journal of Economics, The RAND Corporation, vol. 31(3), pages 395-421, Autumn.
    15. Liran Einav & Amy Finkelstein & Mark R. Cullen, 2010. "Estimating Welfare in Insurance Markets Using Variation in Prices," The Quarterly Journal of Economics, Oxford University Press, vol. 125(3), pages 877-921.
    16. Berkowitz, Daniel & Caner, Mehmet & Fang, Ying, 2008. "Are "Nearly Exogenous Instruments" reliable?," Economics Letters, Elsevier, vol. 101(1), pages 20-23, October.
    17. Pierre-Olivier Gourinchas & Jonathan A. Parker, 2002. "Consumption Over the Life Cycle," Econometrica, Econometric Society, vol. 70(1), pages 47-89, January.
    18. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    19. Gregory S. Crawford & Ali Yurukoglu, 2012. "The Welfare Effects of Bundling in Multichannel Television Markets," American Economic Review, American Economic Association, vol. 102(2), pages 643-685, April.
    20. Greg Kaplan, 2012. "Moving Back Home: Insurance against Labor Market Risk," Journal of Political Economy, University of Chicago Press, vol. 120(3), pages 446-512.
    21. Boris Nikolov & Toni M. Whited, 2014. "Agency Conflicts and Cash: Estimates from a Dynamic Model," Journal of Finance, American Finance Association, vol. 69(5), pages 1883-1921, October.
    22. Timothy G. Conley & Christian B. Hansen & Peter E. Rossi, 2012. "Plausibly Exogenous," The Review of Economics and Statistics, MIT Press, vol. 94(1), pages 260-272, February.
    23. Guggenberger, Patrik, 2012. "On The Asymptotic Size Distortion Of Tests When Instruments Locally Violate The Exogeneity Assumption," Econometric Theory, Cambridge University Press, vol. 28(2), pages 387-421, April.
    24. Ronald L. Goettler & Brett R. Gordon, 2011. "Does AMD Spur Intel to Innovate More?," Journal of Political Economy, University of Chicago Press, vol. 119(6), pages 1141-1200.
    25. Raj Chetty, 2009. "Sufficient Statistics for Welfare Analysis: A Bridge Between Structural and Reduced-Form Methods," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 451-488, May.
    26. Mariacristina De Nardi & Eric French & John B. Jones, 2010. "Why Do the Elderly Save? The Role of Medical Expenses," Journal of Political Economy, University of Chicago Press, vol. 118(1), pages 39-75, February.
    27. Matthew Gentzkow, 2007. "Valuing New Goods in a Model with Complementarity: Online Newspapers," American Economic Review, American Economic Association, vol. 97(3), pages 713-744, June.
    28. Christopher R. Knittel & Konstantinos Metaxoglou, 2014. "Estimation of Random-Coefficient Demand Models: Two Empiricists' Perspective," The Review of Economics and Statistics, MIT Press, vol. 96(1), pages 34-59, March.
    29. Xiaohong Chen & Elie Tamer & Alexander Torgovitsky, 2011. "Sensitivity Analysis in Semiparametric Likelihood Models," Cowles Foundation Discussion Papers 1836, Cowles Foundation for Research in Economics, Yale University.
    30. Matthew Gentzkow & Jesse M. Shapiro & Michael Sinkinson, 2014. "Competition and Ideological Diversity: Historical Evidence from US Newspapers," American Economic Review, American Economic Association, vol. 104(10), pages 3073-3114, October.
    31. Smith, A A, Jr, 1993. "Estimating Nonlinear Time-Series Models Using Simulated Vector Autoregressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 8(S), pages 63-84, Suppl. De.
    32. M. Kate Bundorf & Jonathan Levin & Neale Mahoney, 2012. "Pricing and Welfare in Health Plan Choice," American Economic Review, American Economic Association, vol. 102(7), pages 3214-3248, December.
    33. Michael J. Mazzeo, 2002. "Product Choice and Oligopoly Market Structure," RAND Journal of Economics, The RAND Corporation, vol. 33(2), pages 221-242, Summer.
    34. Gourieroux, C & Monfort, A & Renault, E, 1993. "Indirect Inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 8(S), pages 85-118, Suppl. De.
    35. Amil Petrin, 2002. "Quantifying the Benefits of New Products: The Case of the Minivan," Journal of Political Economy, University of Chicago Press, vol. 110(4), pages 705-729, August.
    36. Robin S. Lee, 2013. "Vertical Integration and Exclusivity in Platform and Two-Sided Markets," American Economic Review, American Economic Association, vol. 103(7), pages 2960-3000, December.
    37. David Berger & Joseph Vavra, 2015. "Consumption Dynamics During Recessions," Econometrica, Econometric Society, vol. 83, pages 101-154, January.
    38. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "An Evaluation of Instrumental Variable Strategies for Estimating the Effects of Catholic Schooling," Journal of Human Resources, University of Wisconsin Press, vol. 40(4), pages 791-821.
    39. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    40. Liran Einav & Amy Finkelstein & Paul Schrimpf, 2013. "The Response of Drug Expenditures to Non-Linear Contract Design: Evidence from Medicare Part D," NBER Working Papers 19393, National Bureau of Economic Research, Inc.
    41. Melanie Morten, 2016. "Temporary Migration and Endogenous Risk Sharing in Village India," NBER Working Papers 22159, National Bureau of Economic Research, Inc.
    42. Sonia Jaffe & E. Glen Weyl, 2013. "The First-Order Approach to Merger Analysis," American Economic Journal: Microeconomics, American Economic Association, vol. 5(4), pages 188-218, November.
    43. repec:feb:framed:0087 is not listed on IDEAS
    44. Jae-On Kim & G. Donald Ferree Jr., 1981. "Standardization in Causal Analysis," Sociological Methods & Research, , vol. 10(2), pages 187-210, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2020. "Transparency in Structural Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 711-722, October.
    2. Friberg, Richard & Romahn, André, 2015. "Divestiture requirements as a tool for competition policy: A case from the Swedish beer market," International Journal of Industrial Organization, Elsevier, vol. 42(C), pages 1-18.
    3. Victor Aguirregabiria & Margaret Slade, 2017. "Empirical models of firms and industries," Canadian Journal of Economics, Canadian Economics Association, vol. 50(5), pages 1445-1488, December.
    4. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2020. "On the Informativeness of Descriptive Statistics for Structural Estimates," Econometrica, Econometric Society, vol. 88(6), pages 2231-2258, November.
    5. Martin Gaynor & Kate Ho & Robert J. Town, 2015. "The Industrial Organization of Health-Care Markets," Journal of Economic Literature, American Economic Association, vol. 53(2), pages 235-284, June.
    6. Matthew Gentzkow & Jesse M. Shapiro & Michael Sinkinson, 2014. "Competition and Ideological Diversity: Historical Evidence from US Newspapers," American Economic Review, American Economic Association, vol. 104(10), pages 3073-3114, October.
    7. Wang, Ao, 2021. "A BLP Demand Model of Product-Level Market Shares with Complementarity," The Warwick Economics Research Paper Series (TWERPS) 1351, University of Warwick, Department of Economics.
    8. Iaria, Alessandro & WANG, Ao, 2020. "Identification and Estimation of Demand for Bundles," CEPR Discussion Papers 14363, C.E.P.R. Discussion Papers.
    9. Gaynor, Martin & Town, Robert J., 2011. "Competition in Health Care Markets," Handbook of Health Economics, in: Mark V. Pauly & Thomas G. Mcguire & Pedro P. Barros (ed.), Handbook of Health Economics, volume 2, chapter 0, pages 499-637, Elsevier.
    10. Bokhari, Farasat A.S. & Mariuzzo, Franco, 2018. "Demand estimation and merger simulations for drugs: Logits v. AIDS," International Journal of Industrial Organization, Elsevier, vol. 61(C), pages 653-685.
    11. Amit Gandhi & Jean-François Houde, 2019. "Measuring Substitution Patterns in Differentiated-Products Industries," NBER Working Papers 26375, National Bureau of Economic Research, Inc.
    12. Adalja, Aaron A., 2018. "The Impact of Voluntary Non-GMO Labeling on Demand in the Ready-to-Eat Cereal Industry," 2018 Annual Meeting, August 5-7, Washington, D.C. 273817, Agricultural and Applied Economics Association.
    13. Timothy J. Layton & Randall P. Ellis & Thomas G. McGuire, 2015. "Assessing Incentives for Adverse Selection in Health Plan Payment Systems," Boston University - Department of Economics - Working Papers Series wp2015-024, Boston University - Department of Economics.
    14. R. Scott Hiller, 2016. "The importance of quality: How music festivals achieved commercial success," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 40(3), pages 309-334, August.
    15. Glenn W. Harrison, 2019. "The behavioral welfare economics of insurance," The Geneva Risk and Insurance Review, Palgrave Macmillan;International Association for the Study of Insurance Economics (The Geneva Association), vol. 44(2), pages 137-175, September.
    16. Liran Einav & Amy Finkelstein, 2017. "Moral Hazard in Health Insurance: What We Know and How We Know It," NBER Working Papers 24055, National Bureau of Economic Research, Inc.
    17. Jorge Rodriguez, 2017. "Understanding the Effects of Income and Child Care Subsidies on Children's Academic Achievement," 2017 Papers pro1077, Job Market Papers.
    18. Ivan Moreno-Torres, 2011. "Generic drugs in Spain: price competition vs. moral hazard," Working Papers XREAP2011-04, Xarxa de Referència en Economia Aplicada (XREAP), revised May 2011.
    19. Christopher R. Knittel & Konstantinos Metaxoglou, 2008. "Estimation of Random Coefficient Demand Models: Challenges, Difficulties and Warnings," NBER Working Papers 14080, National Bureau of Economic Research, Inc.
    20. Kiviet, Jan F., 2020. "Testing the impossible: Identifying exclusion restrictions," Journal of Econometrics, Elsevier, vol. 218(2), pages 294-316.

    More about this item

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:qjecon:v:132:y:2017:i:4:p:1553-1592.. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press or Christopher F. Baum (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.