IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v218y2020i2p247-270.html
   My bibliography  Save this article

Impossible inference in econometrics: Theory and applications

Author

Listed:
  • Bertanha, Marinho
  • Moreira, Marcelo J.

Abstract

This paper studies models in which hypothesis tests have trivial power, that is, power smaller than size. This testing impossibility, or impossibility type A, arises when any alternative is not distinguishable from the null. We also study settings in which it is impossible to have almost surely bounded confidence sets for a parameter of interest. This second type of impossibility (type B) occurs under a condition weaker than the condition for type A impossibility: the parameter of interest must be nearly unidentified. Our theoretical framework connects many existing publications on impossible inference that rely on different notions of topologies to show models are not distinguishable or nearly unidentified. We also derive both types of impossibility using the weak topology induced by convergence in distribution. Impossibility in the weak topology is often easier to prove, it is applicable for many widely-used tests, and it is useful for robust hypothesis testing. We conclude by demonstrating impossible inference in multiple economic applications of models with discontinuity and time-series models.

Suggested Citation

  • Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
  • Handle: RePEc:eee:econom:v:218:y:2020:i:2:p:247-270
    DOI: 10.1016/j.jeconom.2020.04.016
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407620301366
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jeconom.2020.04.016?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kamat, Vishal, 2018. "On Nonparametric Inference In The Regression Discontinuity Design," Econometric Theory, Cambridge University Press, vol. 34(3), pages 694-703, June.
    2. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    3. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    4. Ivan A. Canay & Andres Santos & Azeem M. Shaikh, 2013. "On the Testability of Identification in Some Nonparametric Models With Endogeneity," Econometrica, Econometric Society, vol. 81(6), pages 2535-2559, November.
    5. Sandra E. Black, 1999. "Do Better Schools Matter? Parental Valuation of Elementary Education," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 577-599.
    6. Timothy B. Armstrong & Michal Kolesár, 2018. "Optimal Inference in a Class of Regression Models," Econometrica, Econometric Society, vol. 86(2), pages 655-683, March.
    7. Helena Skyt Nielsen & Torben Sørensen & Christopher Taber, 2010. "Estimating the Effect of Student Aid on College Enrollment: Evidence from a Government Grant Policy Reform," NBER Chapters, in: Income Taxation, Trans-Atlantic Public Economics Seminar (TAPES), pages 185-215, National Bureau of Economic Research, Inc.
    8. Marianne Simonsen & Lars Skipper & Niels Skipper, 2016. "Price Sensitivity of Demand for Prescription Drugs: Exploiting a Regression Kink Design," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(2), pages 320-337, March.
    9. Carolina Caetano, 2015. "A Test of Exogeneity Without Instrumental Variables in Models With Bunching," Econometrica, Econometric Society, vol. 83(4), pages 1581-1600, July.
    10. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    11. Bertanha, Marinho, 2020. "Regression discontinuity design with many thresholds," Journal of Econometrics, Elsevier, vol. 218(1), pages 216-241.
    12. Elise Coudin & Jean-Marie Dufour, 2009. "Finite-sample distribution-free inference in linear median regressions under heteroscedasticity and non-linear dependence of unknown form," Econometrics Journal, Royal Economic Society, vol. 12(s1), pages 19-49, January.
    13. François Gerard & Miikka Rokkanen & Christoph Rothe, 2020. "Bounds on treatment effects in regression discontinuity designs with a manipulated running variable," Quantitative Economics, Econometric Society, vol. 11(3), pages 839-870, July.
    14. Chamberlain, Gary, 1987. "Asymptotic efficiency in estimation with conditional moment restrictions," Journal of Econometrics, Elsevier, vol. 34(3), pages 305-334, March.
    15. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    16. Marinho Bertanha & Guido W. Imbens, 2020. "External Validity in Fuzzy Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 593-612, July.
    17. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    18. Johannes F. Schmieder & Till von Wachter & Stefan Bender, 2012. "The Effects of Extended Unemployment Insurance Over the Business Cycle: Evidence from Regression Discontinuity Estimates Over 20 Years," The Quarterly Journal of Economics, Oxford University Press, vol. 127(2), pages 701-752.
    19. Emmanuel Saez, 2010. "Do Taxpayers Bunch at Kink Points?," American Economic Journal: Economic Policy, American Economic Association, vol. 2(3), pages 180-212, August.
    20. Brian A. Jacob & Lars Lefgren, 2004. "Remedial Education and Student Achievement: A Regression-Discontinuity Analysis," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 226-244, February.
    21. Henrik J. Kleven & Mazhar Waseem, 2013. "Using Notches to Uncover Optimization Frictions and Structural Elasticities: Theory and Evidence from Pakistan," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 669-723.
    22. Donna Feir & Thomas Lemieux & Vadim Marmer, 2016. "Weak Identification in Fuzzy Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(2), pages 185-196, April.
    23. Jean-Marie Dufour, 1997. "Some Impossibility Theorems in Econometrics with Applications to Structural and Dynamic Models," Econometrica, Econometric Society, vol. 65(6), pages 1365-1388, November.
    24. Joseph P. Romano, 2004. "On Non‐parametric Testing, the Uniform Behaviour of the t‐test, and Related Problems," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 31(4), pages 567-584, December.
    25. Diebold, Francis X. & Inoue, Atsushi, 2001. "Long memory and regime switching," Journal of Econometrics, Elsevier, vol. 105(1), pages 131-159, November.
    26. Marcelo J. Moreira, 2003. "A Conditional Likelihood Ratio Test for Structural Models," Econometrica, Econometric Society, vol. 71(4), pages 1027-1048, July.
    27. Johannes F. Schmieder† & Till von Wachter & Stefan Bender, 2011. "The Effects Of Extended Unemployment Insurance Over The Business Cycle: Evidence From Regression Discontinuity Estimates Over Twenty Years," Boston University - Department of Economics - Working Papers Series WP2011-063, Boston University - Department of Economics.
    28. Barbour, A. D. & Utev, Sergey, 1999. "Compound Poisson approximation in total variation," Stochastic Processes and their Applications, Elsevier, vol. 82(1), pages 89-125, July.
    29. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," Review of Economic Studies, Oxford University Press, vol. 79(3), pages 933-959.
    30. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    31. Clive W.J. Granger & Namwon Hyung, 2013. "Occasional Structural Breaks and Long Memory," Annals of Economics and Finance, Society for AEF, vol. 14(2), pages 739-764, November.
    32. Perron, Pierre, 1989. "The Great Crash, the Oil Price Shock, and the Unit Root Hypothesis," Econometrica, Econometric Society, vol. 57(6), pages 1361-1401, November.
    33. Frank Kleibergen, 2005. "Testing Parameters in GMM Without Assuming that They Are Identified," Econometrica, Econometric Society, vol. 73(4), pages 1103-1123, July.
    34. Yingying Dong & Arthur Lewbel, 2015. "Identifying the Effect of Changing the Policy Threshold in Regression Discontinuity Models," The Review of Economics and Statistics, MIT Press, vol. 97(5), pages 1081-1092, December.
    35. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-384, March.
    36. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    37. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 533-575.
    38. William R. Parke, 1999. "What Is Fractional Integration?," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 632-638, November.
    39. Alison L. Gibbs & Francis Edward Su, 2002. "On Choosing and Bounding Probability Metrics," International Statistical Review, International Statistical Institute, vol. 70(3), pages 419-435, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marcelo Moreira & Geert Ridder, 2019. "Efficiency loss of asymptotically efficient tests in an instrumental variables regression," CeMMAP working papers CWP03/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Bugni, Federico A. & Canay, Ivan A., 2021. "Testing continuity of a density via g-order statistics in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 221(1), pages 138-159.
    3. Marinho Bertanha & Andrew H. McCallum & Nathan Seegert, 2021. "Better Bunching, Nicer Notching," Papers 2101.01170, arXiv.org.
    4. Jean-Marie Dufour & Emmanuel Flachaire & Lynda Khalaf & Abdallah Zalghout, 2020. "Identification-robust Inequality Analysis," CIRANO Working Papers 2020s-23, CIRANO.
    5. Marinho Bertanha & EunYi Chung, 2021. "Permutation Tests at Nonparametric Rates," Papers 2102.13638, arXiv.org.
    6. Guido Imbens & Stefan Wager, 2019. "Optimized Regression Discontinuity Designs," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 264-278, May.
    7. Bertanha, Marinho, 2020. "Regression discontinuity design with many thresholds," Journal of Econometrics, Elsevier, vol. 218(1), pages 216-241.
    8. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    2. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    3. Yingying DONG & Ying-Ying LEE & Michael GOU, 2019. "Regression Discontinuity Designs with a Continuous Treatment," Discussion papers 19058, Research Institute of Economy, Trade and Industry (RIETI).
    4. Bertanha, Marinho, 2020. "Regression discontinuity design with many thresholds," Journal of Econometrics, Elsevier, vol. 218(1), pages 216-241.
    5. Tuvaandorj, Purevdorj, 2020. "Regression discontinuity designs, white noise models, and minimax," Journal of Econometrics, Elsevier, vol. 218(2), pages 587-608.
    6. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    7. Chiang, Harold D. & Hsu, Yu-Chin & Sasaki, Yuya, 2019. "Robust uniform inference for quantile treatment effects in regression discontinuity designs," Journal of Econometrics, Elsevier, vol. 211(2), pages 589-618.
    8. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25461, University Library of Munich, Germany.
    9. Martin Huber, 2019. "An introduction to flexible methods for policy evaluation," Papers 1910.00641, arXiv.org.
    10. Villamizar-Villegas, Mauricio & Pinzón-Puerto, Freddy A. & Ruiz-Sánchez, María Alejandra, 2020. "A Comprehensive History of Regression Discontinuity Designs: An Empirical Survey of the last 60 Years," Working papers 38, Red Investigadores de Economía.
    11. Deng, Taotao & Hu, Yukun & Ma, Mulan, 2019. "Regional policy and tourism: A quasi-natural experiment," Annals of Tourism Research, Elsevier, vol. 74(C), pages 1-16.
    12. He, Yang & Bartalotti, Otávio, 2019. "Wild Bootstrap for Fuzzy Regression Discontinuity Designs: Obtaining Robust Bias-Corrected Confidence Intervals," IZA Discussion Papers 12801, Institute of Labor Economics (IZA).
    13. He, Yang & Bartalotti, Otávio, 2019. "Wild bootstrap for fuzzy regression discontinuity designs: obtaining robust bias-corrected confidence intervals," ISU General Staff Papers 201903010800001071, Iowa State University, Department of Economics.
    14. Yu‐Chin Hsu & Shu Shen, 2021. "Testing monotonicity of conditional treatment effects under regression discontinuity designs," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(3), pages 346-366, April.
    15. Bartalotti, Otávio C. & Calhoun, Gray & He, Yang, 2016. "Bootstrap Confidence Intervals for Sharp Regression Discontinuity Designs with the Uniform Kernel," Staff General Research Papers Archive 3394, Iowa State University, Department of Economics.
    16. Guido Imbens & Stefan Wager, 2019. "Optimized Regression Discontinuity Designs," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 264-278, May.
    17. Chen, Heng & Fan, Yanqin, 2019. "Identification and wavelet estimation of weighted ATE under discontinuous and kink incentive assignment mechanisms," Journal of Econometrics, Elsevier, vol. 212(2), pages 476-502.
    18. Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," Review of Economic Studies, Oxford University Press, vol. 85(3), pages 1577-1608.
    19. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.

    More about this item

    Keywords

    Hypothesis tests; Confidence intervals; Weak identification; Regression discontinuity;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:218:y:2020:i:2:p:247-270. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/jeconom .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.