IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2208.00552.html

The Effect of Omitted Variables on the Sign of Regression Coefficients

Author

Listed:
  • Matthew A. Masten
  • Alexandre Poirier

Abstract

We show that, depending on how the impact of omitted variables is measured, it can be substantially easier for omitted variables to flip coefficient signs than to drive them to zero. This behavior occurs with "Oster's delta" (Oster 2019), a widely reported robustness measure. Consequently, any time this measure is large -- suggesting that omitted variables may be unimportant -- a much smaller value reverses the sign of the parameter of interest. We propose a modified measure of robustness to address this concern. We illustrate our results in four empirical applications and two meta-analyses. We implement our methods in the companion Stata module regsensitivity.

Suggested Citation

  • Matthew A. Masten & Alexandre Poirier, 2022. "The Effect of Omitted Variables on the Sign of Regression Coefficients," Papers 2208.00552, arXiv.org, revised Jun 2025.
    • Handle: RePEc:arx:papers:2208.00552
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2208.00552
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. González, Felipe & Miguel, Edward, 2015. "War and local collective action in Sierra Leone: A comment on the use of coefficient stability approaches," Journal of Public Economics, Elsevier, vol. 128(C), pages 30-33.
    2. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    3. Bellows, John & Miguel, Edward, 2009. "War and local collective action in Sierra Leone," Journal of Public Economics, Elsevier, vol. 93(11-12), pages 1144-1157, December.
    4. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    5. Kevin M. Murphy & Robert H. Topel, 1990. "Efficiency Wages Reconsidered: Theory and Evidence," Palgrave Macmillan Books, in: Yoram Weiss & Gideon Fishelson (ed.), Advances in the Theory and Measurement of Unemployment, chapter 8, pages 204-240, Palgrave Macmillan.
    6. Carlos Cinelli & Chad Hazlett, 2020. "Making sense of sensitivity: extending omitted variable bias," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(1), pages 39-67, February.
    7. Emily Oster, 2019. "Unobservable Selection and Coefficient Stability: Theory and Evidence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(2), pages 187-204, April.
    8. Krauth Brian, 2016. "Bounding a Linear Causal Effect Using Relative Correlation Restrictions," Journal of Econometric Methods, De Gruyter, vol. 5(1), pages 117-141, January.
    9. Matthew A. Masten & Alexandre Poirier, 2018. "Identification of Treatment Effects Under Conditional Partial Independence," Econometrica, Econometric Society, vol. 86(1), pages 317-351, January.
    10. Trevor S. Breusch, 1986. "Hypothesis Testing in Unidentified Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 635-651.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Diegert & Matthew A. Masten & Alexandre Poirier, 2022. "Assessing Omitted Variable Bias when the Controls are Endogenous," Papers 2206.02303, arXiv.org, revised Feb 2026.
    2. Matthew A. Masten & Alexandre Poirier & Linqi Zhang, 2024. "Assessing Sensitivity to Unconfoundedness: Estimation and Inference," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(1), pages 1-13, January.
    3. Jeffrey R. Bloem, 2021. "Aspirations and investments in rural Myanmar," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(4), pages 727-752, December.
    4. Leopoldo Fergusson & Carlos A. Molina & James A. Robinson, 2022. "The Weak State Trap," Economica, London School of Economics and Political Science, vol. 89(354), pages 293-331, April.
    5. Sam Jones & Thomas Pave Sohnesen & Neda Trifkovic, 2023. "Educational expansion and shifting private returns to education: Evidence from Mozambique," Journal of International Development, John Wiley & Sons, Ltd., vol. 35(6), pages 1407-1428, August.
    6. Takahashi, Ryo, 2021. "How to stimulate environmentally friendly consumption: Evidence from a nationwide social experiment in Japan to promote eco-friendly coffee," Ecological Economics, Elsevier, vol. 186(C).
    7. Kahsay, Goytom Abraha & Medhin, Haileselassie, 2020. "Leader turnover and forest management outcomes: Micro-level evidence from Ethiopia," World Development, Elsevier, vol. 127(C).
    8. Stéphane Bonhomme & Martin Weidner, 2022. "Minimizing sensitivity to model misspecification," Quantitative Economics, Econometric Society, vol. 13(3), pages 907-954, July.
    9. Pontus af Buren & Jurg Schweri, 2024. "Firms' training processes and their apprentices' education success," Economics of Education Working Paper Series 0225, University of Zurich, Department of Business Administration (IBW).
    10. Daniel Blaseg & Christian Schulze & Bernd Skiera, 2020. "Consumer Protection on Kickstarter," Marketing Science, INFORMS, vol. 39(1), pages 211-233, January.
    11. Alexis H. Villacis & Jeffrey R. Alwang & Victor Barrera & Juan Dominguez, 2022. "Prices, specialty varieties, and postharvest practices: Insights from cacao value chains in Ecuador," Agribusiness, John Wiley & Sons, Ltd., vol. 38(2), pages 426-458, April.
    12. Michael Babington & Sebastian J. Goerg & Carl Kitchens, 2020. "Do Tournaments With Superstars Encourage or Discourage Competition?," Journal of Sports Economics, , vol. 21(1), pages 44-63, January.
    13. Betts, Alexander & Flinder Stierna, Maria & Omata, Naohiko & Sterck, Olivier, 2023. "Refugees welcome? Inter-group interaction and host community attitude formation," World Development, Elsevier, vol. 161(C).
    14. Betts,Alexander Milton Stedman & Stierna,Maria Flinder & Omata,Naohiko & Sterck,Olivier Christian Brigitte, 2022. "Social Cohesion and Refugee-Host Interactions : Evidence from East Africa," Policy Research Working Paper Series 9917, The World Bank.
    15. Deepankar Basu, 2023. "Formal Covariate Benchmarking to Bound Omitted Variable Bias," Papers 2306.10562, arXiv.org.
    16. Bensch, Gunther & Kluve, Jochen & Stöterau, Jonathan, 2021. "The market-based dissemination of energy-access technologies as a business model for rural entrepreneurs: Evidence from Kenya," Resource and Energy Economics, Elsevier, vol. 66(C).
    17. Hollard, Guillaume & Sene, Omar, 2020. "What drives the quality of schools in Africa? Disentangling social capital and ethnic divisions," Economics of Education Review, Elsevier, vol. 74(C).
    18. Pietro Emilio Spini, 2021. "Robustness, Heterogeneous Treatment Effects and Covariate Shifts," Papers 2112.09259, arXiv.org, revised Aug 2024.
    19. Cemal Eren Arbatlı & Quamrul H. Ashraf & Oded Galor & Marc Klemp, 2020. "Diversity and Conflict," Econometrica, Econometric Society, vol. 88(2), pages 727-797, March.
    20. Clemens, Michael A. & Montenegro, Claudio & Pritchett, Lant, 2016. "Bounding the Price Equivalent of Migration Barriers," IZA Discussion Papers 9789, IZA Network @ LISER.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2208.00552. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.