IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/19054.html
   My bibliography  Save this paper

Unobservable Selection and Coefficient Stability: Theory and Validation

Author

Listed:
  • Emily Oster

Abstract

A common heuristic for evaluating robustness of results to omitted variable bias is to look at coefficient movements after inclusion of controls. This heuristic is informative only if selection on observables is proportional to selection on unobservables. I formalize this link, drawing on theory in Altonji, Elder and Taber (2005) and show how, with this assumption, coefficient movements, along with movements in R-squared values, can be used to calculate omitted variable bias. I discuss empirical implementation and describe a formal bounding argument to replace the coefficient movement heuristic. I show two validation exercises suggesting that this bounding argument would perform well empirically. I discuss application of this procedure to a large set of publications in economics, and use evidence from randomized studies to draw guidelines as to appropriate bounding values.

Suggested Citation

  • Emily Oster, 2013. "Unobservable Selection and Coefficient Stability: Theory and Validation," NBER Working Papers 19054, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:19054
    Note: EH LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w19054.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    2. Tamer, Elie, 2010. "Partial Identification in Econometrics," Scholarly Articles 34728615, Harvard University Department of Economics.
    3. Kevin M. Murphy & Robert H. Topel, 1990. "Efficiency Wages Reconsidered: Theory and Evidence," Palgrave Macmillan Books, in: Yoram Weiss & Gideon Fishelson (ed.), Advances in the Theory and Measurement of Unemployment, chapter 8, pages 204-240, Palgrave Macmillan.
    4. Robert Jensen, 2012. "Do Labor Market Opportunities Affect Young Women's Work and Family Decisions? Experimental Evidence from India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(2), pages 753-792.
    5. Douglas Almond & Bhashkar Mazumder, 2011. "Health Capital and the Prenatal Environment: The Effect of Ramadan Observance during Pregnancy," American Economic Journal: Applied Economics, American Economic Association, vol. 3(4), pages 56-85, October.
    6. Robert J. LaLonde, 1984. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," Working Papers 563, Princeton University, Department of Economics, Industrial Relations Section..
    7. Nicola Lacetera & Devin G. Pope & Justin R. Sydnor, 2012. "Heuristic Thinking and Limited Attention in the Car Market," American Economic Review, American Economic Association, vol. 102(5), pages 2206-2236, August.
    8. Mazumder Bhashkar, 2011. "Family and Community Influences on Health and Socioeconomic Status: Sibling Correlations Over the Life Course," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 11(3), pages 1-23, September.
    9. Pierre-André Chiappori & Sonia Oreffice & Climent Quintana-Domeque, 2012. "Fatter Attraction: Anthropometric and Socioeconomic Matching on the Marriage Market," Journal of Political Economy, University of Chicago Press, vol. 120(4), pages 659-695.
    10. Atif Mian & Amir Sufi, 2009. "The Consequences of Mortgage Credit Expansion: Evidence from the U.S. Mortgage Default Crisis," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(4), pages 1449-1496.
    11. Nathan Nunn & Leonard Wantchekon, 2011. "The Slave Trade and the Origins of Mistrust in Africa," American Economic Review, American Economic Association, vol. 101(7), pages 3221-3252, December.
    12. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    13. Elie Tamer, 2010. "Partial Identification in Econometrics," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 167-195, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Firpo, Sergio & Galvao, Antonio F. & Parker, Thomas, 2023. "Uniform inference for value functions," Journal of Econometrics, Elsevier, vol. 235(2), pages 1680-1699.
    2. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    3. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LLC, vol. 7(3), pages 334-350, September.
    4. Alloush, Mo & Bloem, Jeffrey R., 2022. "Neighborhood violence, poverty, and psychological well-being," Journal of Development Economics, Elsevier, vol. 154(C).
    5. Greenstone, Michael & Gayer, Ted, 2009. "Quasi-experimental and experimental approaches to environmental economics," Journal of Environmental Economics and Management, Elsevier, vol. 57(1), pages 21-44, January.
    6. Myoung-jae Lee, 2007. "Difference in Generalized-Differences with Panel Data: Effects of Moving from Private to Public School on Test Scores," Discussion Paper Series 0721, Institute of Economic Research, Korea University.
    7. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    8. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    9. Matthew A. Masten & Alexandre Poirier & Linqi Zhang, 2024. "Assessing Sensitivity to Unconfoundedness: Estimation and Inference," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(1), pages 1-13, January.
    10. Juan Carlos Escanciano & Lin Zhu, 2013. "Set inferences and sensitivity analysis in semiparametric conditionally identified models," CeMMAP working papers 55/13, Institute for Fiscal Studies.
    11. Todd, Annika & Cappers, Peter & Spurlock, C. Anna & Jin, Ling, 2019. "Spillover as a cause of bias in baseline evaluation methods for demand response programs," Applied Energy, Elsevier, vol. 250(C), pages 344-357.
    12. Arild Aakvik & Tor Helge Holmås & Egil Kjerstad, 2024. "Health status, life expectancy and early claiming of pension," SN Business & Economics, Springer, vol. 4(4), pages 1-20, April.
    13. Helena Holmlund & Olmo Silva, 2014. "Targeting Noncognitive Skills to Improve Cognitive Outcomes: Evidence from a Remedial Education Intervention," Journal of Human Capital, University of Chicago Press, vol. 8(2), pages 126-160.
    14. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2006. "Teacher-Student Matching and the Assessment of Teacher Effectiveness," Journal of Human Resources, University of Wisconsin Press, vol. 41(4).
    15. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    16. Paul Diegert & Matthew A. Masten & Alexandre Poirier, 2022. "Assessing Omitted Variable Bias when the Controls are Endogenous," Papers 2206.02303, arXiv.org, revised Jul 2023.
    17. Chakravarty, Shubha & Lundberg, Mattias & Nikolov, Plamen & Zenker, Juliane, 2019. "Vocational training programs and youth labor market outcomes: Evidence from Nepal," Journal of Development Economics, Elsevier, vol. 136(C), pages 71-110.
    18. David Card & Pablo Ibarrarán & Ferdinando Regalia & David Rosas-Shady & Yuri Soares, 2011. "The Labor Market Impacts of Youth Training in the Dominican Republic," Journal of Labor Economics, University of Chicago Press, vol. 29(2), pages 267-300.
    19. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
    20. Calvi, Rossella & Mantovanelli, Federico G., 2018. "Long-term effects of access to health care: Medical missions in colonial India," Journal of Development Economics, Elsevier, vol. 135(C), pages 285-303.

    More about this item

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • I1 - Health, Education, and Welfare - - Health
    • I12 - Health, Education, and Welfare - - Health - - - Health Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:19054. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.