IDEAS home Printed from
   My bibliography  Save this article

Using Generalizability Theory to Examine Scoring Reliability and Variability of Judging Panels in Skating Competitions


  • Huang Jinyan

    (Niagara University)

  • Foote Chandra J

    (Niagara University)


Generalizability (G-) theory is increasingly being used by assessment professionals in a variety of assessment contexts, especially those involving performance assessments. This study expands the use of G-theory and its possible application to highly competitive sporting events that involve judging panels. Specifically, using G-theory, this study examined both the scoring variability and reliability of all four events of the 2004 World Figure Skating Championships. For each event, it examined the independent sources of score variation contributing to the variability of scores assigned by the raters (judges), and the score reliability (or generalizability coefficient). For the Ice Dancing event, an additional examination of the impact of treating rater and rating as different facets of G-theory studies on the score reliability was conducted. The results show that rater/rating was identified as a component of concern in all events. For the Singles events, in particular, the percentages of the variance for rater (judge) reached approximately 10 percent of the total variance, indicating that there were serious concerns about the scoring reliability. The generalizability coefficients for the events of Mens Singles and Ladies Singles were both 0.93. However, the generalizability coefficients for Pairs and Ice Dancing were 0.99 and 0.98, respectively, indicating that the scores for these two events were extremely reliable. Using rater vs. rating in the Ice Dancing scores appeared to result in very little difference.

Suggested Citation

  • Huang Jinyan & Foote Chandra J, 2011. "Using Generalizability Theory to Examine Scoring Reliability and Variability of Judging Panels in Skating Competitions," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 7(3), pages 1-23, July.
  • Handle: RePEc:bpj:jqsprt:v:7:y:2011:i:3:n:16

    Download full text from publisher

    File URL:
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Alamar Benjamin C, 2010. "Measuring Risk in NFL Playcalling," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 6(2), pages 1-9, April.
    2. Alamar Benjamin C, 2006. "The Passing Premium Puzzle," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 2(4), pages 1-10, October.
    Full references (including those not matched with items on IDEAS)

    More about this item


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:jqsprt:v:7:y:2011:i:3:n:16. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Peter Golla). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.