IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i9p1547-d808559.html
   My bibliography  Save this article

Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model

Author

Listed:
  • Daniel Garcia Iglesias

    (Arrhythmia Unit, Cardiology Department, Hospital Universitario Central de Asturias, 33011 Oviedo, Spain
    Instituto de Investigación Sanitaria del Principado de Asturias, 33011 Oviedo, Spain)

Abstract

Propensity Score Matching (PSM) is a useful method to reduce the impact of Treatment-Selection Bias in the estimation of causal effects in observational studies. After matching, the PSM significantly reduces the sample under investigation, which may lead to other possible biases (due to overfitting, excess of covariation or a reduced number of observations). In this sense, we wanted to analyze the behavior of this PSM compared with other widely used methods to deal with non-comparable groups, such as the Multivariate Regression Model (MRM). Monte Carlo Simulations are made to construct groups with different effects in order to compare the behavior of PSM and MRM estimating these effects. In addition, the Treatment Selection Bias reduction for the PSM is calculated. With the PSM a reduction in the Treatment Selection Bias is achieved (0.983 [0.982, 0.984]), with a reduction in the Relative Real Treatment Effect Estimation Error (0.216 [0.2, 0.232]), but despite this bias reduction and estimation error reduction, the MRM reduces this estimation error significantly more than the PSM (0.539 [0.522, 0.556], p < 0.001). In addition, the PSM leads to a 30% reduction in the sample. This loss of information derived from the matching process may lead to another not known bias and thus to the inaccuracy of the effect estimation compared with the MRM.

Suggested Citation

  • Daniel Garcia Iglesias, 2022. "Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model," Mathematics, MDPI, vol. 10(9), pages 1-8, May.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:9:p:1547-:d:808559
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/9/1547/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/9/1547/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ho, Daniel & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2011. "MatchIt: Nonparametric Preprocessing for Parametric Causal Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i08).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chervier, Colas & Le Velly, Gwenolé & Ezzine-de-Blas, Driss, 2019. "When the Implementation of Payments for Biodiversity Conservation Leads to Motivation Crowding-out: A Case Study From the Cardamoms Forests, Cambodia," Ecological Economics, Elsevier, vol. 156(C), pages 499-510.
    2. Moritz Flubacher & George Sheldon & Adrian Müller, 2015. "Comparison of the Economic Performance between Organic and Conventional Dairy Farms in the Swiss Mountain Region Using Matching and Stochastic Frontier Analysis," Journal of Socio-Economics in Agriculture (Until 2015: Yearbook of Socioeconomics in Agriculture), Swiss Society for Agricultural Economics and Rural Sociology, vol. 7(1), pages 76-84.
    3. Rigdon, Joseph & Berkowitz, Seth A. & Seligman, Hilary K. & Basu, Sanjay, 2017. "Re-evaluating associations between the Supplemental Nutrition Assistance Program participation and body mass index in the context of unmeasured confounders," Social Science & Medicine, Elsevier, vol. 192(C), pages 112-124.
    4. Finocchiaro Castro, Massimo & Guccio, Calogero & Rizzo, Ilde, 2023. "How "one-size-fits-all" public works contract does it better? An assessment of infrastructure provision in Italy," EconStor Preprints 270729, ZBW - Leibniz Information Centre for Economics.
    5. Lo, A. W.-T. & Houston, D., 2018. "How do compact, accessible, and walkable communities promote gender equality in spatial behavior?," Journal of Transport Geography, Elsevier, vol. 68(C), pages 42-54.
    6. Stjepan Srhoj & Michael Lapinski & Janette Walde, 2019. "Size matters? Impact evaluation of business development grants on SME performance," Working Papers 2019-14, Faculty of Economics and Statistics, Universität Innsbruck.
    7. Jiaming Zeng & Michael F. Gensheimer & Daniel L. Rubin & Susan Athey & Ross D. Shachter, 2022. "Uncovering interpretable potential confounders in electronic medical records," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    8. Dong, Hongwei, 2017. "Rail-transit-induced gentrification and the affordability paradox of TOD," Journal of Transport Geography, Elsevier, vol. 63(C), pages 1-10.
    9. Chen, Shanting & Mallory, Allen B., 2021. "The effect of racial discrimination on mental and physical health: A propensity score weighting approach," Social Science & Medicine, Elsevier, vol. 285(C).
    10. Katie Devenish & Sébastien Desbureaux & Simon Willcock & Julia P. G. Jones, 2022. "On track to achieve no net loss of forest at Madagascar’s biggest mine," Nature Sustainability, Nature, vol. 5(6), pages 498-508, June.
    11. Changjun Gu & Pei Zhao & Qiong Chen & Shicheng Li & Lanhui Li & Linshan Liu & Yili Zhang, 2020. "Forest Cover Change and the Effectiveness of Protected Areas in the Himalaya since 1998," Sustainability, MDPI, vol. 12(15), pages 1-24, July.
    12. Arne Lauer & Samantha L. Speroni & Myoung Choi & Xiao Da & Christine Duncan & Siobhan McCarthy & Vijai Krishnan & Cole A. Lusk & David Rohde & Mikkel Bo Hansen & Jayashree Kalpathy-Cramer & Daniel J. , 2023. "Hematopoietic stem-cell gene therapy is associated with restored white matter microvascular function in cerebral adrenoleukodystrophy," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    13. Maekawa, Wakako, 2024. "United Nations peacekeeping operations and multilateral foreign aid: Credibility of good governance," World Development, Elsevier, vol. 176(C).
    14. Mei-Cheng Wang & Yuxin Zhu, 2022. "Bias correction via outcome reassignment for cross-sectional data with binary disease outcome," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(4), pages 659-674, October.
    15. Schneeberger, Andres R. & Huber, Christian G. & Lang, Undine E. & Muenzenmaier, Kristina H. & Castille, Dorothy & Jaeger, Matthias & Seixas, Azizi & Sowislo, Julia & Link, Bruce G., 2017. "Effects of assisted outpatient treatment and health care services on psychotic symptoms," Social Science & Medicine, Elsevier, vol. 175(C), pages 152-160.
    16. Ferentinos, Konstantinos & Gibberd, Alex & Guin, Benjamin, 2023. "Stranded houses? The price effect of a minimum energy efficiency standard," Energy Economics, Elsevier, vol. 120(C).
    17. Injeong Shim & Hiroyuki Kuwahara & NingNing Chen & Mais O. Hashem & Lama AlAbdi & Mohamed Abouelhoda & Hong-Hee Won & Pradeep Natarajan & Patrick T. Ellinor & Amit V. Khera & Xin Gao & Fowzan S. Alkur, 2023. "Clinical utility of polygenic scores for cardiometabolic disease in Arabs," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    18. Giuseppina Spano & Mario Elia & Onofrio Cappelluti & Giuseppe Colangelo & Vincenzo Giannico & Marina D’Este & Raffaele Lafortezza & Giovanni Sanesi, 2021. "Is Experience the Best Teacher? Knowledge, Perceptions, and Awareness of Wildfire Risk," IJERPH, MDPI, vol. 18(16), pages 1-12, August.
    19. Suss, Joel & Bholat, David & Gillespie, Alex & Reader, Tom, 2021. "Organisational culture and bank risk," Bank of England working papers 912, Bank of England.
    20. de Rassenfosse, Gaétan & Pellegrino, Gabriele & Raiteri, Emilio, 2024. "Do patents enable disclosure? Evidence from the invention secrecy act," International Journal of Industrial Organization, Elsevier, vol. 92(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:9:p:1547-:d:808559. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.