IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v82y2015icp1-18.html
   My bibliography  Save this article

Weighted kappa statistic for clustered matched-pair ordinal data

Author

Listed:
  • Yang, Zhao
  • Zhou, Ming

Abstract

As an important extension of the regular kappa statistic, the weighted kappa statistic has been widely used to assess the agreement between two procedures for independent matched-pair ordinal data. For clustered matched-pair ordinal data, based on the delta method and sampling techniques, a non-parametric variance estimator for the weighted kappa statistic is proposed without within-cluster correlation structure or distributional assumptions. The results of an extensive Monte Carlo simulation study demonstrate that the proposed weighted kappa statistic provides consistent estimation, and the proposed variance estimator behaves reasonably well for at least a moderately large number of clusters (e.g., K≥50). Compared with the variance estimator ignoring dependence within a cluster, the proposed variance estimator performs better in maintaining the nominal coverage probability when the intra-cluster correlation is fair (ρ≥0.3), with more pronounced improvement when ρ is further increased. Moreover, under the general analysis of variance setting with systematic variability between procedures and clusters being included as a component of total variation, the equivalence between weighted kappa statistic and intraclass correlation coefficient is established. To illustrate the practical application of the proposed estimator, two real medical research data examples of clustered matched-pair ordinal data are analyzed, including an agreement study to compare two methods for assessing cervical ectopy, and a physician–patients data example from the Enhancing Communication and HIV Outcomes study.

Suggested Citation

  • Yang, Zhao & Zhou, Ming, 2015. "Weighted kappa statistic for clustered matched-pair ordinal data," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 1-18.
    • Handle: RePEc:eee:csdana:v:82:y:2015:i:c:p:1-18
    DOI: 10.1016/j.csda.2014.08.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947314002369
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2014.08.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jun-Mo Nam, 2000. "Interval Estimation of the Kappa Coefficient with Binary Classification and an Equal Marginal Probability Model," Biometrics, The International Biometric Society, vol. 56(2), pages 583-585, June.
    2. Huiman X. Barnhart & John M. Williamson, 2002. "Weighted Least-Squares Approach for Comparing Correlated Kappa," Biometrics, The International Biometric Society, vol. 58(4), pages 1012-1019, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang, Zhao & Zhou, Ming, 2015. "Kappa statistic for clustered physician–patients polytomous data," Computational Statistics & Data Analysis, Elsevier, vol. 87(C), pages 1-17.
    2. Sandipta Debanshi & Swades Pal, 2020. "Assessing gully erosion susceptibility in Mayurakshi river basin of eastern India," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 22(2), pages 883-914, February.
    3. Matthijs J. Warrens, 2021. "Kappa coefficients for dichotomous-nominal classifications," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(1), pages 193-208, March.
    4. Matthijs J. Warrens & Bunga C. Pratiwi, 2016. "Kappa Coefficients for Circular Classifications," Journal of Classification, Springer;The Classification Society, vol. 33(3), pages 507-522, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kilem Gwet, 2008. "Variance Estimation of Nominal-Scale Inter-Rater Reliability with Random Selection of Raters," Psychometrika, Springer;The Psychometric Society, vol. 73(3), pages 407-430, September.
    2. Allan Dormer & Guangyong Zou, 2002. "Interval Estimation for a Difference Between Intraclass Kappa Statistics," Biometrics, The International Biometric Society, vol. 58(1), pages 209-215, March.
    3. Tsai, Miao-Yu & Wang, Jung-Feng & Wu, Jia-Ling, 2011. "Generalized estimating equations with model selection for comparing dependent categorical agreement data," Computational Statistics & Data Analysis, Elsevier, vol. 55(7), pages 2354-2362, July.
    4. Hung‐Mo Lin & John M. Williamson & Stuart R. Lipsitz, 2003. "Calculating power for the comparison of dependent κ‐coefficients," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 52(4), pages 391-404, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:82:y:2015:i:c:p:1-18. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.