IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2206.02303.html
   My bibliography  Save this paper

Assessing Omitted Variable Bias when the Controls are Endogenous

Author

Listed:
  • Paul Diegert
  • Matthew A. Masten
  • Alexandre Poirier

Abstract

Omitted variables are one of the most important threats to the identification of causal effects. Several widely used approaches, including Oster (2019), assess the impact of omitted variables on empirical conclusions by comparing measures of selection on observables with measures of selection on unobservables. These approaches either (1) assume the omitted variables are uncorrelated with the included controls, an assumption that is often considered strong and implausible, or (2) use a method called residualization to avoid this assumption. In our first contribution, we develop a framework for objectively comparing sensitivity parameters. We use this framework to formally prove that the residualization method generally leads to incorrect conclusions about robustness. In our second contribution, we then provide a new approach to sensitivity analysis that avoids this critique, allows the omitted variables to be correlated with the included controls, and lets researchers calibrate sensitivity parameters by comparing the magnitude of selection on observables with the magnitude of selection on unobservables as in previous methods. We illustrate our results in an empirical study of the effect of historical American frontier life on modern cultural beliefs. Finally, we implement these methods in the companion Stata module regsensitivity for easy use in practice.

Suggested Citation

  • Paul Diegert & Matthew A. Masten & Alexandre Poirier, 2022. "Assessing Omitted Variable Bias when the Controls are Endogenous," Papers 2206.02303, arXiv.org, revised Jul 2023.
  • Handle: RePEc:arx:papers:2206.02303
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2206.02303
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 71, Elsevier.
    3. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    4. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    5. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    6. Matthew A. Masten & Alexandre Poirier & Linqi Zhang, 2024. "Assessing Sensitivity to Unconfoundedness: Estimation and Inference," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(1), pages 1-13, January.
    7. Carlos Cinelli & Chad Hazlett, 2020. "Making sense of sensitivity: extending omitted variable bias," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(1), pages 39-67, February.
    8. Emily Oster, 2019. "Unobservable Selection and Coefficient Stability: Theory and Evidence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(2), pages 187-204, April.
    9. Karim Chalak, 2019. "Identification of average effects under magnitude and sign restrictions on confounding," Quantitative Economics, Econometric Society, vol. 10(4), pages 1619-1657, November.
    10. Tabellini, Marco, 2020. "Gifts of the Immigrants, Woes of the Natives: Lessons from the Age of Mass Migration," CEPR Discussion Papers 14317, C.E.P.R. Discussion Papers.
    11. Krauth Brian, 2016. "Bounding a Linear Causal Effect Using Relative Correlation Restrictions," Journal of Econometric Methods, De Gruyter, vol. 5(1), pages 117-141, January.
    12. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    13. Matthew A. Masten & Alexandre Poirier, 2018. "Identification of Treatment Effects Under Conditional Partial Independence," Econometrica, Econometric Society, vol. 86(1), pages 317-351, January.
    14. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    15. Marco Tabellini, 2020. "Gifts of the Immigrants, Woes of the Natives: Lessons from the Age of Mass Migration," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 87(1), pages 454-486.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kaicheng Chen & Kyoo il Kim, 2024. "Identification of Nonseparable Models with Endogenous Control Variables," Papers 2401.14395, arXiv.org.
    2. João Martins & Linda Veiga & Bruno Fernandes, 2023. "Are electronic government innovations helpful to deter corruption? Evidence from across the world," Economics and Politics, Wiley Blackwell, vol. 35(3), pages 1177-1203, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthew A. Masten & Alexandre Poirier & Linqi Zhang, 2024. "Assessing Sensitivity to Unconfoundedness: Estimation and Inference," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(1), pages 1-13, January.
    2. Matthew A. Masten & Alexandre Poirier, 2022. "The Effect of Omitted Variables on the Sign of Regression Coefficients," Papers 2208.00552, arXiv.org, revised Feb 2023.
    3. Graziella Bertocchi & Arcangelo Dimico & Gian Luca Tedeschi, 2022. "Strangers and Foreigners: Trust and Attitudes toward Citizenship," CHILD Working Papers Series 100 JEL Classification: J, Centre for Household, Income, Labour and Demographic Economics (CHILD) - CCA.
    4. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    5. Graziella Bertocchi & Angelo Dimico & Gian Luca Tedeschi, 2022. "Strangers and Foreigners: Trust and Attitudes toward Citizenship," Department of Economics 0200, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    6. Shusen Qi & Steven Ongena & Hua Cheng, 2022. "Working with women, do men get all the credit?," Small Business Economics, Springer, vol. 59(4), pages 1427-1447, December.
    7. Colnet Bénédicte & Josse Julie & Varoquaux Gaël & Scornet Erwan, 2022. "Causal effect on a target population: A sensitivity analysis to handle missing covariates," Journal of Causal Inference, De Gruyter, vol. 10(1), pages 372-414, January.
    8. Alloush, Mo & Bloem, Jeffrey R., 2022. "Neighborhood violence, poverty, and psychological well-being," Journal of Development Economics, Elsevier, vol. 154(C).
    9. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    10. Yu Yan & Shusen Qi, 2021. "I Know What I Need: Optimization of Bribery," Journal of Business Ethics, Springer, vol. 174(2), pages 311-332, November.
    11. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    12. Victor Chernozhukov & Carlos Cinelli & Whitney Newey & Amit Sharma & Vasilis Syrgkanis, 2021. "Long Story Short: Omitted Variable Bias in Causal Machine Learning," Papers 2112.13398, arXiv.org, revised Nov 2023.
    13. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    14. Simone Moriconi & Giovanni Peri & Riccardo Turati, 2022. "Are Immigrants more Left leaning than Natives?," NBER Working Papers 30523, National Bureau of Economic Research, Inc.
    15. Duncan Chaplin & Arif Mamun & Ali Protik & John Schurrer & Divya Vohra & Kristine Bos & Hannah Burak & Laura Meyer & Anca Dumitrescu & Christopher Ksoll & Thomas Cook, "undated". "Grid Electricity Expansion in Tanzania by MCC: Findings from a Rigorous Impact Evaluation, Final Report," Mathematica Policy Research Reports 144768f69008442e96369195e, Mathematica Policy Research.
    16. Guido W. Imbens, 2015. "Matching Methods in Practice: Three Examples," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 373-419.
    17. Pietro Emilio Spini, 2021. "Robustness, Heterogeneous Treatment Effects and Covariate Shifts," Papers 2112.09259, arXiv.org.
    18. Tymon S{l}oczy'nski, 2018. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," Papers 1810.01576, arXiv.org, revised May 2020.
    19. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    20. Noy, Shakked & Sin, Isabelle, 2021. "The effects of neighbourhood and workplace income comparisons on subjective wellbeing," Journal of Economic Behavior & Organization, Elsevier, vol. 185(C), pages 918-945.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2206.02303. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.