IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0144962.html
   My bibliography  Save this article

Evaluating Propensity Score Methods in a Quasi-Experimental Study of the Impact of Menu-Labeling

Author

Listed:
  • Stephanie L Mayne
  • Brian K Lee
  • Amy H Auchincloss

Abstract

Background: Quasi-experimental studies of menu labeling have found mixed results for improving diet. Differences between experimental groups can hinder interpretation. Propensity scores are an increasingly common method to improve covariate balance, but multiple methods exist and the improvements associated with each method have rarely been compared. In this re-analysis of the impact of menu labeling, we compare multiple propensity score methods to determine which methods optimize balance between experimental groups. Methods: Study participants included adult customers who visited full-service restaurants with menu labeling (treatment) and without (control). We compared the balance between treatment groups obtained by four propensity score methods: 1) 1:1 nearest neighbor matching (NN), 2) augmented 1:1 NN (using caliper of 0.2 and an exact match on an imbalanced covariate), 3) full matching, and 4) inverse probability weighting (IPW). We then evaluated the treatment effect on differences in nutrients purchased across the different methods. Results: 1:1 NN resulted in worse balance than the original unmatched sample (average standardized absolute mean distance [ASAM]: 0.185 compared to 0.171). Augmented 1:1 NN improved balance (ASAM: 0.038) but resulted in a large reduction in sample size. Full matching and IPW improved balance over the unmatched sample without a reduction in sample size (ASAM: 0.049 and 0.031, respectively). Menu labeling was associated with decreased calories, fat, sodium and carbohydrates in the unmatched analysis. Results were qualitatively similar in the propensity score matched/weighted models. Conclusions: While propensity scores offer an increasingly popular tool to improve causal inference, choosing the correct method can be challenging. Our results emphasize the benefit of examining multiple methods to ensure results are consistent, and considering approaches beyond the most popular method of 1:1 NN matching.

Suggested Citation

  • Stephanie L Mayne & Brian K Lee & Amy H Auchincloss, 2015. "Evaluating Propensity Score Methods in a Quasi-Experimental Study of the Impact of Menu-Labeling," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-12, December.
    • Handle: RePEc:plo:pone00:0144962
    DOI: 10.1371/journal.pone.0144962
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0144962
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0144962&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0144962?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peter C. Austin, 2009. "The Relative Ability of Different Propensity Score Methods to Balance Measured Covariates Between Treated and Untreated Subjects in Observational Studies," Medical Decision Making, , vol. 29(6), pages 661-677, November.
    2. Denis Conniffe & Vanessa Gash & Philip J. O'Connell, 2000. "Evaluating State Programmes - “Natural Experiments” and Propensity Scores," The Economic and Social Review, Economic and Social Studies, vol. 31(4), pages 283-308.
    3. Ben B. Hansen, 2004. "Full Matching in an Observational Study of Coaching for the SAT," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 609-618, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andre M. N. Renzaho & Stanley Chitekwe & Wen Chen & Sanjay Rijal & Thakur Dhakal & Pradiumna Dahal, 2017. "The Synergetic Effect of Cash Transfers for Families, Child Sensitive Social Protection Programs, and Capacity Building for Effective Social Protection on Children’s Nutritional Status in Nepal," IJERPH, MDPI, vol. 14(12), pages 1-22, December.
    2. Yahui Wang & Liangjie Xin & Xiubin Li & Jianzhong Yan, 2016. "Impact of Land Use Rights Transfer on Household Labor Productivity: A Study Applying Propensity Score Matching in Chongqing, China," Sustainability, MDPI, vol. 9(1), pages 1-18, December.
    3. Hebe N Gouda & Andrew Hodge & Raoul Bermejo III & Willibald Zeck & Eliana Jimenez-Soto, 2016. "The Impact of Healthcare Insurance on the Utilisation of Facility-Based Delivery for Childbirth in the Philippines," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-15, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martín-García, Jaime & Gómez-Limón, José A. & Arriaza, Manuel, 2024. "Conversion to organic farming: Does it change the economic and environmental performance of fruit farms?," Ecological Economics, Elsevier, vol. 220(C).
    2. Zhenzhen Xu & John D. Kalbfleisch, 2013. "Repeated Randomization and Matching in Multi-Arm Trials," Biometrics, The International Biometric Society, vol. 69(4), pages 949-959, December.
    3. Orlando Jiménez, 2005. "Innovation-Oriented Environmental Regulations: Direct versus Indirect Regulations; an Empirical Analysis of Small and Medium-Sized Enterprises in Chile," Environment and Planning A, , vol. 37(4), pages 723-750, April.
    4. Kelly, Elish & McGuinness, Seamus & O'Connell, Philip J., 2011. "What Can Active Labour Market Policies Do?," Papers EC1, Economic and Social Research Institute (ESRI).
    5. Aggarwal, Reena & Erel, Isil & Stulz, Rene M. & Williamson, Rohan, 2006. "Do U.S. Firms Have the Best Corporate Governance? A Cross-Country Examination of the Relation between Corporate Governance and Shareholder Wealth," Working Paper Series 2006-25, Ohio State University, Charles A. Dice Center for Research in Financial Economics.
    6. Zhenzhen Xu & John D. Kalbfleisch, 2010. "Propensity Score Matching in Randomized Clinical Trials," Biometrics, The International Biometric Society, vol. 66(3), pages 813-823, September.
    7. Pierluigi Montalbano & Silvia Nenci & Laura Dell'Agostino, 2022. "A non-parametric assessment of the effects of the Euro on GVC trade," International Economics, CEPII research center, issue 172, pages 56-76.
    8. Reena Aggarwal & Isil Erel & René Stulz & Rohan Williamson, 2010. "Differences in Governance Practices between U.S. and Foreign Firms: Measurement, Causes, and Consequences," The Review of Financial Studies, Society for Financial Studies, vol. 23(3), pages 3131-3169, March.
    9. Alberto Abadie & Guido W. Imbens, 2012. "A Martingale Representation for Matching Estimators," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 833-843, June.
    10. Motoi Kusadokoro & Ai Hasegawa, 2017. "The Influence of Internal Migration on Migrant Children’s School Enrolment and Work in Turkey," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 29(2), pages 348-368, April.
    11. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    12. Jason Lyall, 2008. "Does Indiscriminate Violence Incite Insurgent Attacks? Evidence from a Natural Experiment," HiCN Working Papers 44, Households in Conflict Network.
    13. Casey A. Klofstad & Benjamin G. Bishin, 2014. "Do Social Ties Encourage Immigrant Voters to Participate in Other Campaign Activities?," Social Science Quarterly, Southwestern Social Science Association, vol. 95(2), pages 295-310, June.
    14. Anna Kovner & Chenyang Wei, 2012. "The private premium in public bonds," Staff Reports 553, Federal Reserve Bank of New York.
    15. Tom Fangyun Tan & Serguei Netessine, 2020. "At Your Service on the Table: Impact of Tabletop Technology on Restaurant Performance," Management Science, INFORMS, vol. 66(10), pages 4496-4515, October.
    16. Joel F. Houston & Liangliang Jiang & Chen Lin & Yue Ma, 2014. "Political Connections and the Cost of Bank Loans," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 52(1), pages 193-243, March.
    17. Marion Ravit & Andrainolo Ravalihasy & Martine Audibert & Valery Ridde & Emmanuel Bonnet & Bertille Raffalli & Flore-Apolline Roy & Anais N’landu & Alexandre Dumont, 2020. "The impact of the obstetrical risk insurance scheme in Mauritania on maternal healthcare utilization: a propensity score matching analysis," Post-Print hal-02509190, HAL.
    18. Loyalka, Prashant & Zakharov, Andrey, 2016. "Does shadow education help students prepare for college? Evidence from Russia," International Journal of Educational Development, Elsevier, vol. 49(C), pages 22-30.
    19. Helmreich, James E. & Pruzek, Robert M., 2009. "PSAgraphics: An R Package to Support Propensity Score Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 29(i06).
    20. Andre Rossi Oliveira, 2024. "Evaluating the Short-term Causal Effect of Early Alert on Student Performance," Research in Higher Education, Springer;Association for Institutional Research, vol. 65(7), pages 1395-1419, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0144962. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.