IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp3998.html

New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators

Author

Listed:
  • Busso, Matias

    (Inter-American Development Bank)

  • DiNardo, John

    (University of Michigan)

  • McCrary, Justin

    (University of California, Berkeley)

Abstract

Currently available asymptotic results in the literature suggest that matching estimators have higher variance than reweighting estimators. The extant literature comparing the finite sample properties of matching to specific reweighting estimators, however, has concluded that reweighting performs far worse than even the simplest matching estimator. We resolve this puzzle. We show that the findings from the finite sample analyses are not inconsistent with asymptotic analysis, but are very specific to particular choices regarding the implementation of reweighting, and fail to generalize to settings likely to be encountered in actual empirical practice. In the DGPs studied here, reweighting typically outperforms propensity score matching.

Suggested Citation

  • Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, IZA Network @ LISER.
    • Handle: RePEc:iza:izadps:dp3998
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp3998.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Martin Biewen, 2001. "Measuring the Effects of Socio-Economic Variables on the Income Distribution: An Application to the East German Transition Process," The Review of Economics and Statistics, MIT Press, vol. 83(1), pages 185-190, February.
    2. John W. Budd & Brian P. McCall, 2001. "The Grocery Stores Wage Distribution: A Semi-Parametric Analysis of the Role of Retailing and Labor Market Institutions," ILR Review, Cornell University, ILR School, vol. 54(2A), pages 484-501, March.
    3. Oaxaca, Ronald, 1973. "Male-Female Wage Differentials in Urban Labor Markets," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 14(3), pages 693-709, October.
    4. Justin McCrary, 2007. "The Effect of Court-Ordered Hiring Quotas on the Composition and Quality of Police," American Economic Review, American Economic Association, vol. 97(1), pages 318-353, March.
    5. Newey, Whitney K, 1990. "Semiparametric Efficiency Bounds," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 5(2), pages 99-135, April-Jun.
    6. repec:bla:obuest:v:60:y:1998:i:4:p:509-28:a is not listed on IDEAS
    7. Brian D. Bell & Michael K. Pitt, 1998. "Trade Union Decline and the Distribution of Wages in the UK: Evidence from Kernel Density Estimation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 60(4), pages 509-528, November.
    8. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    9. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    10. Bell, Brian D & Pitt, Michael K, 1998. "Trade Union Decline and the Distribution of Wages in the UK: Evidence from Kernel Density Estimation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 60(4), pages 509-528, November.
    11. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(2), pages 261-294.
    12. Alan S. Blinder, 1973. "Wage Discrimination: Reduced Form and Structural Estimates," Journal of Human Resources, University of Wisconsin Press, vol. 8(4), pages 436-455.
    13. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    14. Dehejia, Rajeev H., 2005. "Program evaluation as a decision problem," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 141-173.
    15. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthias Draheim & Peter Schanbacher & Ruben Seiberlich, 2021. "On the effectiveness of case management for people with disabilities," Journal for Labour Market Research, Springer;Institute for Employment Research/ Institut für Arbeitsmarkt- und Berufsforschung (IAB), vol. 55(1), pages 1-16, December.
    2. Taisuke Otsu & Mengshan Xu, 2022. "Isotonic propensity score matching," STICERD - Econometrics Paper Series 623, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    3. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    4. Mengshan Xu & Taisuke Otsu, 2022. "Isotonic propensity score matching," Papers 2207.08868, arXiv.org, revised Jan 2025.
    5. Kitagawa, Toru & Muris, Chris, 2016. "Model averaging in semiparametric estimation of treatment effects," Journal of Econometrics, Elsevier, vol. 193(1), pages 271-289.
    6. Halbert White & Karim Chalak, 2013. "Identification and Identification Failure for Treatment Effects Using Structural Systems," Econometric Reviews, Taylor & Francis Journals, vol. 32(3), pages 273-317, November.
    7. Gevrek, Z. Eylem & Seiberlich, Ruben R., 2014. "Semiparametric decomposition of the gender achievement gap: An application for Turkey," Labour Economics, Elsevier, vol. 31(C), pages 27-44.
    8. Ying-Ying Lee, 2014. "Partial Mean Processes with Generated Regressors: Continuous Treatment Effects and Nonseparable Models," Economics Series Working Papers 706, University of Oxford, Department of Economics.
    9. Flores, Carlos A. & Flores-Lagunes, Alfonso, 2009. "Identification and Estimation of Causal Mechanisms and Net Effects of a Treatment under Unconfoundedness," IZA Discussion Papers 4237, Institute of Labor Economics (IZA).
    10. Frolich, Markus, 2007. "Nonparametric IV estimation of local average treatment effects with covariates," Journal of Econometrics, Elsevier, vol. 139(1), pages 35-75, July.
    11. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    12. Yukitoshi Matsushita & Taisuke Otsu & Keisuke Takahata, 2022. "Estimating density ratio of marginals to joint: Applications to causal inference," STICERD - Econometrics Paper Series 619, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    13. Rahul Singh & Liyuan Xu & Arthur Gretton, 2020. "Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves," Papers 2010.04855, arXiv.org, revised Oct 2022.
    14. Lin, Zhexiao & Han, Fang, 2025. "On regression-adjusted imputation estimators of average treatment effects," Journal of Econometrics, Elsevier, vol. 251(C).
    15. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2006. "Moving the Goalposts: Addressing Limited Overlap in the Estimation of Average Treatment Effects by Changing the Estimand," NBER Technical Working Papers 0330, National Bureau of Economic Research, Inc.
    16. Ham, John C. & Li, Xianghong & Reagan, Patricia B., 2011. "Matching and semi-parametric IV estimation, a distance-based measure of migration, and the wages of young men," Journal of Econometrics, Elsevier, vol. 161(2), pages 208-227, April.
    17. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    18. James J. Heckman, 2008. "The Principles Underlying Evaluation Estimators with an Application to Matching," Annals of Economics and Statistics, GENES, issue 91-92, pages 9-73.
    19. Zeqin Liu & Zongwu Cai & Ying Fang & Ming Lin, 2019. "Statistical Analysis and Evaluation of Macroeconomic Policies: A Selective Review," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 201904, University of Kansas, Department of Economics, revised Mar 2019.
    20. Cattaneo, Matias D., 2010. "Efficient semiparametric estimation of multi-valued treatment effects under ignorability," Journal of Econometrics, Elsevier, vol. 155(2), pages 138-154, April.

    More about this item

    Keywords

    ;
    ;
    ;

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp3998. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.