IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0241.html
   My bibliography  Save this paper

When to Control for Covariates? Panel-Asymptotic Results for Estimates of Treatment Effects

Author

Listed:
  • Joshua D. Angrist
  • Jinyong Hahn

Abstract

The problem of how to control for covariates is endemic in evaluation research. Covariate-matching provides an appealing control strategy, but with continuous or high-dimensional covariate vectors, exact matching may be impossible or involve small cells. Matching observations that have the same propensity score produces unbiased estimates of causal effects whenever covariate-matching does, and also has an attractive dimension-reducing property. On the other hand, conventional asymptotic arguments show that covariate-matching is (asymptotically) more efficient that propensity score-matching. This is because the usual asymptotic sequence has cell sizes growing to infinity, with no benefit from reducing the number of cells. Here, we approximate the large sample behavior of difference matching estimators using a panel-style asymptotic sequence with fixed cell sizes and the number of cells increasing to infinity. Exact calculations in simple examples and Monte Carlo evidence suggests this generates a substantially improved approximation to actual finite-sample distributions. Under this sequence, propensity-score-matching is most likely to dominate exact matching when cell sizes are small, the explanatory power of the covariates conditional on the propensity score is low, and/or the probability of treatment is close to zero or one. Finally, we introduce a random-effects type combination estimator that provides finite-sample efficiency gains over both covariate-matching and propensity-score-matching.

Suggested Citation

  • Joshua D. Angrist & Jinyong Hahn, 1999. "When to Control for Covariates? Panel-Asymptotic Results for Estimates of Treatment Effects," NBER Technical Working Papers 0241, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberte:0241
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0241.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    2. Hausman, Jerry A & Taylor, William E, 1981. "Panel Data and Unobservable Individual Effects," Econometrica, Econometric Society, vol. 49(6), pages 1377-1398, November.
    3. Swamy, P A V B, 1970. "Efficient Inference in a Random Coefficient Regression Model," Econometrica, Econometric Society, vol. 38(2), pages 311-323, March.
    4. Angrist, Joshua D & Krueger, Alan B, 1995. "Split-Sample Instrumental Variables Estimates of the Return to Schooling," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 225-235, April.
    5. Bekker, Paul A, 1994. "Alternative Approximations to the Distributions of Instrumental Variable Estimators," Econometrica, Econometric Society, vol. 62(3), pages 657-681, May.
    6. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    7. Angus Deaton & Christina Paxson, 1998. "Economies of Scale, Household Size, and the Demand for Food," Journal of Political Economy, University of Chicago Press, vol. 106(5), pages 897-930, October.
    8. Chamberlain, Gary & Imbens, Guido, 1996. "Hierarchical Bayes Models with Many Instrumental Variables," Scholarly Articles 3221489, Harvard University Department of Economics.
    9. Douglas Staiger & James H. Stock, 1997. "Instrumental Variables Regression with Weak Instruments," Econometrica, Econometric Society, vol. 65(3), pages 557-586, May.
    10. Chamberlain, Gary, 1984. "Panel data," Handbook of Econometrics,in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 22, pages 1247-1318 Elsevier.
    11. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    12. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-660, November.
    13. Joshua D. Angrist & Alan B. Krueger, 1993. "Split Sample Instrumental Variables," Working Papers 699, Princeton University, Department of Economics, Industrial Relations Section..
    14. Card, David & Sullivan, Daniel G, 1988. "Measuring the Effect of Subsidized Training Programs on Movements in and out of Employment," Econometrica, Econometric Society, vol. 56(3), pages 497-530, May.
    15. Chamberlain, Gary, 1987. "Asymptotic efficiency in estimation with conditional moment restrictions," Journal of Econometrics, Elsevier, vol. 34(3), pages 305-334, March.
    16. Maddala, G S, 1971. "The Use of Variance Components Models in Pooling Cross Section and Time Series Data," Econometrica, Econometric Society, vol. 39(2), pages 341-358, March.
    17. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    18. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    19. Chamberlain, Gary & Griliches, Zvi, 1975. "Unobservables with a Variance-Components Structure: Ability, Schooling, and the Economic Success of Brothers," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 16(2), pages 422-449, June.
    20. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," Review of Economic Studies, Oxford University Press, vol. 65(2), pages 261-294.
    21. Chamberlain, Gary, 1992. "Efficiency Bounds for Semiparametric Regression," Econometrica, Econometric Society, vol. 60(3), pages 567-596, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Angrist, Joshua & Lavy, Victor, 2002. "The Effect of High School Matriculation Awards: Evidence from Randomized Trials," CEPR Discussion Papers 3827, C.E.P.R. Discussion Papers.
    2. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    3. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    4. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    5. Jochen Kluve & Hartmut Lehmann & Christoph M. Schmidt, 2000. "Disentangling Treatment Effects of Polish Active Labour Market Policies: Evidence from Matched Samples," CERT Discussion Papers 0007, Centre for Economic Reform and Transformation, Heriot Watt University.
    6. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    7. Foster, E. Michael & Stephens, Robert & Krivelyova, Anna & Gamfi, Phyllis, 2007. "Can system integration improve mental health outcomes for children and youth?," Children and Youth Services Review, Elsevier, vol. 29(10), pages 1301-1319, October.
    8. Jochen Kluve & Boris Augurzky, 2007. "Assessing the performance of matching algorithms when selection into treatment is strong," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(3), pages 533-557.
    9. Fajnzylber, Pablo & Maloney, William F. & Rojas, Gabriel V. Montes, 2006. "Releasing constraints to growth or pushing on a string ? the impact of credit, training, business associations, and taxes on the performance of Mexican micro-firms," Policy Research Working Paper Series 3807, The World Bank.
    10. Angrist, Joshua & Lavy, Victor, 2004. "The Effect of High Stakes High School Achievement Awards: Evidence from a School-Centered Randomized Trial," IZA Discussion Papers 1146, Institute for the Study of Labor (IZA).
    11. Joshua Angrist & Victor Lavy, 2003. "Achievement awards for high school matriculation: Evidence from randomized trials," Natural Field Experiments 00202, The Field Experiments Website.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0241. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.