Matching, Covariate Balance, and Bias in Estimated Treatment Effect: A Monte Carlo Simulation Analysis

This chapter examines the effectiveness of five weighting and matching techniques, including propensity score matching (PSM), in improving covariate balance and reducing bias when estimating treatment effects in finite-sample situations through Monte Carlo simulations. King and Nielsen (2019) argue that pruning observations based on PSM with 1-to-1 greedy matching can worsen, rather than improve, covariate balance and increase bias in treatment effect estimates. In our simulations, we observed this phenomenon not only in PSM with 1-to-1 greedy matching but also in other covariate balancing techniques that King and Nielsen (2019) recommend as better matching methods, i.e., Mahalanobis distance matching (MDM) and coarsened exact matching (CEM). Across all weighting and matching techniques and data-generating processes examined in this study, our findings indicate that matching and weighting under extreme caliper or cut-point settings do not improve covariate balance. In addition, once covariate balance is substantially improved, the estimated bias tends to increase slightly as balance continues to improve. Moreover, our simulation results suggest that OLS with appropriately specified covariates reduces selection bias as effectively as the other weighting and matching methods. These findings highlight the importance of avoiding a one-size-fits-all approach and of carefully identifying the appropriate nonexperimental estimator for a given sample by thoroughly examining the characteristics of the available data.