On the Failure of the Bootstrap for Matching Estimators
Matching estimators are widely used for the evaluation of programs or treatments. Often researchers use bootstrapping methods for inference. However, no formal justification for the use of the bootstrap has been provided. Here we show that the bootstrap is in general not valid, even in the simple case with a single continuous covariate when the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias. Due to the extreme non-smoothness of nearest neighbor matching, the standard conditions for the bootstrap are not satisfied, leading the bootstrap variance to diverge from the actual variance. Simulations confirm the difference between actual and nominal coverage rates for bootstrap confidence intervals predicted by the theoretical calculations. To our knowledge, this is the first example of a root-N consistent and asymptotically normal estimator for which the bootstrap fails to work.
|Date of creation:||Jun 2006|
|Date of revision:|
|Publication status:||published as -Abadie, Alberto and Guido W. Imbens. "On the Failure of the Bootstrap for Matching Estimators." Econometrica 76, 6 (2008): 1537-1557.|
|Contact details of provider:|| Postal: National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.|
Web page: http://www.nber.org
More information through EDIRC
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Jeffrey Smith & Petra Todd, 2003.
"Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?,"
University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers
20035, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
- A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
- Roberto Agodini & Mark Dynarski, 2004. "Are Experiments the Only Option? A Look at Dropout Prevention Programs," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 180-194, February.
- James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
- Emanuela Galasso & Martin Ravallion, 2004.
"Social Protection in a Crisis: Argentina's Plan Jefes y Jefas,"
World Bank Economic Review,
World Bank Group, vol. 18(3), pages 367-399.
- Galasso, Emanuela & Ravallion, Martin, 2003. "Social protection in a crisis - Argentina's Plan Jefes y Jefas," Policy Research Working Paper Series 3165, The World Bank.
- Rajeev H. Dehejia & Sadek Wahba, 1998.
"Propensity Score Matching Methods for Non-experimental Causal Studies,"
NBER Working Papers
6829, National Bureau of Economic Research, Inc.
- Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
- Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, 01.
- Whitney Newey & Frank Windmeijer, 2005. "GMM with many weak moment conditions," CeMMAP working papers CWP18/05, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
- Jalan, Jyotsna & Ravallion, Martin, 1999. "Income gains to the poor from workfare - estimates for Argentina's TRABAJAR Program," Policy Research Working Paper Series 2149, The World Bank.
- Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
- Sascha O. Becker & Andrea Ichino, 2002. "Estimation of average treatment effects based on propensity scores," Stata Journal, StataCorp LP, vol. 2(4), pages 358-377, November.
- Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
- Alberto Abadie & David Drukker & Jane Leber Herr & Guido W. Imbens, 2004. "Implementing matching estimators for average treatment effects in Stata," Stata Journal, StataCorp LP, vol. 4(3), pages 290-311, September.
- Michael Lechner, 2005.
"Some practical issues in the evaluation of heterogeneous labour market programmes by matching methods,"
Labor and Demography
- Michael Lechner, 2002. "Some practical issues in the evaluation of heterogeneous labour market programmes by matching methods," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 165(1), pages 59-82.
- Patrick A . Puhani, 2002. "Advantage through Training in Poland? A Microeconometric Evaluation of the Employment Effects of Training and Job Subsidy Programmes," LABOUR, CEIS, vol. 16(3), pages 569-608, 09.
- Menno Pradhan & Laura B. Rawlings, 2002. "The Impact and Targeting of Social Infrastructure Investments: Lessons from the Nicaraguan Social Fund," World Bank Economic Review, World Bank Group, vol. 16(2), pages 275-295, August.
When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0325. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()
If references are entirely missing, you can add them using this form.