IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp9346.html
   My bibliography  Save this paper

Simple Tests for Selection Bias: Learning More from Instrumental Variables

Author

Listed:
  • Black, Dan A.

    (University of Chicago)

  • Joo, Joonhwi

    (University of Chicago)

  • LaLonde, Robert J.

    (Harris School, University of Chicago)

  • Smith, Jeffrey A.

    (University of Wisconsin-Madison)

  • Taylor, Evan J.

    (University of Arizona)

Abstract

We provide simple tests for selection on unobserved variables in the Vytlacil-Imbens-Angrist framework for Local Average Treatment Effects. The tests allow researchers not only to test for selection on either or both of the treated and untreated outcomes, but also to assess the magnitude of the selection effect. The tests are quite simple; undergraduates after an introductory econometrics class should be able to implement these tests. We illustrate our tests with two empirical applications: the impact of children on female labor supply from Angrist and Evans (1998) and the training of adult women from the Job Training Partnership Act (JTPA) experiment.

Suggested Citation

  • Black, Dan A. & Joo, Joonhwi & LaLonde, Robert J. & Smith, Jeffrey A. & Taylor, Evan J., 2015. "Simple Tests for Selection Bias: Learning More from Instrumental Variables," IZA Discussion Papers 9346, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp9346
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp9346.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
    2. Joshua D. Angrist, 2004. "Treatment effect heterogeneity in theory and practice," Economic Journal, Royal Economic Society, vol. 114(494), pages 52-83, March.
    3. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    4. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    5. Joo, Joonhwi & LaLonde, Robert J., 2014. "Testing for Selection Bias," IZA Discussion Papers 8455, Institute of Labor Economics (IZA).
    6. A. D. Roy, 1951. "Some Thoughts On The Distribution Of Earnings," Oxford Economic Papers, Oxford University Press, vol. 3(2), pages 135-146.
    7. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    8. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    9. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    10. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    11. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124.
    12. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    13. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
    14. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    15. Angrist, Joshua D & Evans, William N, 1998. "Children and Their Parents' Labor Supply: Evidence from Exogenous Variation in Family Size," American Economic Review, American Economic Association, vol. 88(3), pages 450-477, June.
    16. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    17. Marinho Bertanha & Guido W. Imbens, 2020. "External Validity in Fuzzy Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 593-612, July.
    18. Curtis Eberwein & John C. Ham & Robert J. Lalonde, 1997. "The Impact of Being Offered and Receiving Classroom Training on the Employment Histories of Disadvantaged Women: Evidence from Experimental Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 655-682.
    19. Howard S. Bloom & Larry L. Orr & Stephen H. Bell & George Cave & Fred Doolittle & Winston Lin & Johannes M. Bos, 1997. "The Benefits and Costs of JTPA Title II-A Programs: Key Findings from the National Job Training Partnership Act Study," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 549-576.
    20. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    21. Richard Blundell & Lorraine Dearden & Barbara Sianesi, 2005. "Evaluating the effect of education on earnings: models, methods and results from the National Child Development Survey," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(3), pages 473-512, July.
    22. Heckman, James J, 1996. "Randomization as an Instrumental Variable: Notes," The Review of Economics and Statistics, MIT Press, vol. 78(2), pages 336-341, May.
    23. Dan A. Black & Seth G. Sanders & Evan J. Taylor & Lowell J. Taylor, 2015. "The Impact of the Great Migration on Mortality of African Americans: Evidence from the Deep South," American Economic Review, American Economic Association, vol. 105(2), pages 477-503, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Martin Huber, 2019. "An introduction to flexible methods for policy evaluation," Papers 1910.00641, arXiv.org.
    2. Martini, Alberto & Rettore, Enrico & Barbetta, Gianpaolo, 2017. "The Impact of Traineeships on the Employment of the Mentally Ill: The Role of Partial Compliance," IZA Discussion Papers 10582, Institute of Labor Economics (IZA).
    3. Steven J. Haider & Melvin Stephens Jr., 2020. "Correcting for Misclassified Binary Regressors Using Instrumental Variables," NBER Working Papers 27797, National Bureau of Economic Research, Inc.
    4. Amanda E Kowalski, 2023. "Behaviour within a Clinical Trial and Implications for Mammography Guidelines," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 432-462.
    5. Johansen, Eva Rye, 2021. "Relative age for grade and adolescent risky health behavior," Journal of Health Economics, Elsevier, vol. 76(C).
    6. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    7. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    8. Anne Sofie Tegner Anker & Jennifer L. Doleac & Rasmus Landersø, 2021. "The Effects of DNA Databases on the Deterrence and Detection of Offenders," American Economic Journal: Applied Economics, American Economic Association, vol. 13(4), pages 194-225, October.
    9. Martin Huber & Jannis Kueck, 2022. "Testing the identification of causal effects in observational data," Papers 2203.15890, arXiv.org, revised Jun 2023.
    10. Brunello, Giorgio & Rocco, Lorenzo, 2016. "Is Childcare Bad for the Mental Health of Grandparents? Evidence from SHARE," IZA Discussion Papers 10022, Institute of Labor Economics (IZA).
    11. Nicolas Apfel & Julia Hatamyar & Martin Huber & Jannis Kueck, 2024. "Learning control variables and instruments for causal analysis in observational data," Papers 2407.04448, arXiv.org.
    12. Giorgio Brunello & Lorenzo Rocco, 2019. "Grandparents in the blues. The effect of childcare on grandparents’ depression," Review of Economics of the Household, Springer, vol. 17(2), pages 587-613, June.
    13. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 49(3), pages 871-905, August.
    14. Martin Huber, 2024. "An Introduction to Causal Discovery," Papers 2407.08602, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    2. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    3. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    4. Baum-Snow, Nathaniel & Ferreira, Fernando, 2015. "Causal Inference in Urban and Regional Economics," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 3-68, Elsevier.
    5. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    6. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    7. James J. Heckman, 2008. "The Principles Underlying Evaluation Estimators with an Application to Matching," Annals of Economics and Statistics, GENES, issue 91-92, pages 9-73.
    8. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
    9. Eliasson, Kent, 2006. "The Role of Ability in Estimating the Returns to College Choice: New Swedish Evidence," Umeå Economic Studies 691, Umeå University, Department of Economics.
    10. Gevrek, Z. Eylem & Seiberlich, Ruben R., 2014. "Semiparametric decomposition of the gender achievement gap: An application for Turkey," Labour Economics, Elsevier, vol. 31(C), pages 27-44.
    11. Gonzalo Nunez-Chaim & Henry G. Overman & Capucine Riom, 2024. "Does subsidising business advice improve firm performance? Evidence from a large RCT," CEP Discussion Papers dp1977, Centre for Economic Performance, LSE.
    12. Richard Blundell & Lorraine Dearden & Barbara Sianesi, 2003. "Evaluating the impact of education on earnings in the UK: Models, methods and results from the NCDS," IFS Working Papers W03/20, Institute for Fiscal Studies.
    13. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    14. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    15. Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2024. "Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," Journal of Human Resources, University of Wisconsin Press, vol. 59(4), pages 1244-1283.
    16. Mazzutti, Caio Cícero Toledo Piza da Costa, 2016. "Three essays on the causal impacts of child labour laws in Brazil," Economics PhD Theses 0616, Department of Economics, University of Sussex Business School.
    17. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    18. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2013. "Are Some Degrees Worth More than Others? Evidence from college admission cutoffs in Chile," NBER Working Papers 19241, National Bureau of Economic Research, Inc.
    19. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    20. Zhao, Zhong, 2008. "Sensitivity of propensity score methods to the specifications," Economics Letters, Elsevier, vol. 98(3), pages 309-319, March.

    More about this item

    Keywords

    selection; local average treatment effects; instrumental variables;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • J01 - Labor and Demographic Economics - - General - - - Labor Economics: General
    • J08 - Labor and Demographic Economics - - General - - - Labor Economics Policies

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp9346. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.