IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v47y2022i4p438-458.html

Statistical Inference for G-indices of Agreement

Author

Listed:
  • Douglas G. Bonett

    (Department of Psychology, University of California, Santa Cruz, CA, USA)

Abstract

The limitations of Cohen’s κ are reviewed and an alternative G -index is recommended for assessing nominal-scale agreement. Maximum likelihood estimates, standard errors, and confidence intervals for a two-rater G -index are derived for one-group and two-group designs. A new G -index of agreement for multirater designs is proposed. Statistical inference methods for some important special cases of the multirater design also are derived. G -index meta-analysis methods are proposed and can be used to combine and compare agreement across two or more populations. Closed-form sample-size formulas to achieve desired confidence interval precision are proposed for two-rater and multirater designs. R functions are given for all results.

Suggested Citation

  • Douglas G. Bonett, 2022. "Statistical Inference for G-indices of Agreement," Journal of Educational and Behavioral Statistics, , vol. 47(4), pages 438-458, August.
    • Handle: RePEc:sae:jedbes:v:47:y:2022:i:4:p:438-458
    DOI: 10.3102/10769986221088561
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986221088561
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986221088561?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Douglas Bonett & Thomas Wright, 2000. "Sample size requirements for estimating pearson, kendall and spearman correlations," Psychometrika, Springer;The Psychometric Society, vol. 65(1), pages 23-28, March.
    2. V. Flack & A. Afifi & P. Lachenbruch & H. Schouten, 1988. "Sample size determinations for the two rater kappa statistic," Psychometrika, Springer;The Psychometric Society, vol. 53(3), pages 321-325, September.
    3. Allan Dormer & Guangyong Zou, 2002. "Interval Estimation for a Difference Between Intraclass Kappa Statistics," Biometrics, The International Biometric Society, vol. 58(1), pages 209-215, March.
    4. Price, Robert M. & Bonett, Douglas G., 2004. "An improved confidence interval for a linear function of binomial proportions," Computational Statistics & Data Analysis, Elsevier, vol. 45(3), pages 449-456, April.
    5. Matthijs Warrens, 2010. "A Formal Proof of a Paradox Associated with Cohen’s Kappa," Journal of Classification, Springer;The Classification Society, vol. 27(3), pages 322-332, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McMahon, James M. & Pouget, Enrique R. & Tortu, Stephanie, 2006. "A guide for multilevel modeling of dyadic data with binary outcomes using SAS PROC NLMIXED," Computational Statistics & Data Analysis, Elsevier, vol. 50(12), pages 3663-3680, August.
    2. Audrey Helen Linden & Thomas V Pollet & Johannes Hönekopp, 2024. "Publication bias in psychology: A closer look at the correlation between sample size and effect size," PLOS ONE, Public Library of Science, vol. 19(2), pages 1-15, February.
    3. Li, Zhitao & Tang, Jinjun & Zhao, Chuyun & Gao, Fan, 2023. "Improved centrality measure based on the adapted PageRank algorithm for urban transportation multiplex networks," Chaos, Solitons & Fractals, Elsevier, vol. 167(C).
    4. S. Divya & Prabu Christopher B, 2025. "The Role of Supervisor Support and Change Information in Determining Behavioral Attitudes Toward Change in an Academic Setting: Empirical Validation of Intra-Organizational Dynamics Using IPMA Analysis," SAGE Open, , vol. 15(3), pages 21582440251, August.
    5. Hothorn Ludwig A & Sill Martin & Schaarschmidt Frank, 2010. "Evaluation of Incidence Rates in Pre-Clinical Studies Using a Williams-Type Procedure," The International Journal of Biostatistics, De Gruyter, vol. 6(1), pages 1-19, April.
    6. Qiang Tu & Han He & Xiaomin Lai & Chuan Jiang & Zhanji Zheng, 2024. "Identifying Critical Links in Degradable Road Networks Using a Traffic Demand-Based Indicator," Sustainability, MDPI, vol. 16(18), pages 1-20, September.
    7. Elina Tsigeman & Sebastian Silas & Klaus Frieler & Maxim Likhanov & Rebecca Gelding & Yulia Kovas & Daniel Müllensiefen, 2022. "The Jack and Jill Adaptive Working Memory Task: Construction, Calibration and Validation," PLOS ONE, Public Library of Science, vol. 17(1), pages 1-29, January.
    8. Casu, Barbara & Ferrari, Alessandra & Girardone, Claudia & Wilson, John O.S., 2016. "Integration, productivity and technological spillovers: Evidence for eurozone banking industries," European Journal of Operational Research, Elsevier, vol. 255(3), pages 971-983.
    9. Boaventura, Joao Mauricio & Carnaúba, A.A.C. & Todeva, Emanuela & Azevedo, A.C. & Armando, Eduardo, 2016. "Governance Structures and Trust: a Study of Real Estate Networks," MPRA Paper 76785, University Library of Munich, Germany.
    10. Christophe Croux & Catherine Dehon, 2010. "Influence functions of the Spearman and Kendall correlation measures," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 19(4), pages 497-515, November.
    11. Sven Baehre & Michele O’Dwyer & Lisa O’Malley & Nick Lee, 2022. "The use of Net Promoter Score (NPS) to predict sales growth: insights from an empirical investigation," Journal of the Academy of Marketing Science, Springer, vol. 50(1), pages 67-84, January.
    12. José Antonio Roldán-Nofuentes & Tulsi Sagar Sheth & José Fernando Vera-Vera, 2024. "Hypothesis Test to Compare Two Paired Binomial Proportions: Assessment of 24 Methods," Mathematics, MDPI, vol. 12(2), pages 1-23, January.
    13. Shakila Dada & Kirsty Bastable & Santoshi Halder, 2020. "The Role of Social Support in Participation Perspectives of Caregivers of Children with Intellectual Disabilities in India and South Africa," IJERPH, MDPI, vol. 17(18), pages 1-18, September.
    14. M. Álvarez Hernández & A. Martín Andrés & I. Herranz Tejedor, 2016. "One-sided asymptotic inferences for a proportion," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(9), pages 1738-1752, July.
    15. Krishna K. Saha & Debaraj Sen & Chun Jin, 2012. "Profile likelihood-based confidence interval for the dispersion parameter in count data," Journal of Applied Statistics, Taylor & Francis Journals, vol. 39(4), pages 765-783, August.
    16. repec:plo:pone00:0028071 is not listed on IDEAS
    17. Huan Tong & Jian Kang, 2021. "Relationship between urban development patterns and noise complaints in England," Environment and Planning B, , vol. 48(6), pages 1632-1649, July.
    18. Smith, Laurence G. & Westaway, Sally & Mullender, Samantha & Ghaley, Bhim Bahadur & Xu, Ying & Lehmann, Lisa Mølgaard & Pisanelli, Andrea & Russo, Giuseppe & Borek, Robert & Wawer, Rafał & Borzęcka, M, 2022. "Assessing the multidimensional elements of sustainability in European agroforestry systems," Agricultural Systems, Elsevier, vol. 197(C).
    19. Hamdi El Asli & Lakhmaiss Hamid & Afif Zineb & Azeroual Mohamed, 2024. "Impact of, Human Capital, Economic Factors, Energy Consumption, and Urban Growth on Environmental Sustainability in Morocco: An ARDL Approach," International Journal of Energy Economics and Policy, Econjournals, vol. 14(2), pages 656-668, March.
    20. Jingjing Zhu & Xiaoying Xia & Qianqian Wu & Shiyao Zou & Yan Li, 2023. "Callous-Unemotional Traits and Social Adjustment among Chinese Preschoolers: The Moderating Role of Teacher-Child Relationship," IJERPH, MDPI, vol. 20(4), pages 1-15, February.
    21. Jugurta Bouidghaghen & Laurence Moreau & Katia Beauchêne & Romain Chapuis & Nathalie Mangel & Llorenç Cabrera‐Bosquet & Claude Welcker & Matthieu Bogard & François Tardieu, 2023. "Robotized indoor phenotyping allows genomic prediction of adaptive traits in the field," Nature Communications, Nature, vol. 14(1), pages 1-14, December.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:47:y:2022:i:4:p:438-458. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.