IDEAS home Printed from
MyIDEAS: Log in (now much improved!) to save this article

Using Generalizability Theory to Examine Scoring Reliability and Variability of Judging Panels in Skating Competitions

Listed author(s):
  • Huang Jinyan

    (Niagara University)

  • Foote Chandra J

    (Niagara University)

Registered author(s):

    Generalizability (G-) theory is increasingly being used by assessment professionals in a variety of assessment contexts, especially those involving performance assessments. This study expands the use of G-theory and its possible application to highly competitive sporting events that involve judging panels. Specifically, using G-theory, this study examined both the scoring variability and reliability of all four events of the 2004 World Figure Skating Championships. For each event, it examined the independent sources of score variation contributing to the variability of scores assigned by the raters (judges), and the score reliability (or generalizability coefficient). For the Ice Dancing event, an additional examination of the impact of treating rater and rating as different facets of G-theory studies on the score reliability was conducted. The results show that rater/rating was identified as a component of concern in all events. For the Singles events, in particular, the percentages of the variance for rater (judge) reached approximately 10 percent of the total variance, indicating that there were serious concerns about the scoring reliability. The generalizability coefficients for the events of Mens Singles and Ladies Singles were both 0.93. However, the generalizability coefficients for Pairs and Ice Dancing were 0.99 and 0.98, respectively, indicating that the scores for these two events were extremely reliable. Using rater vs. rating in the Ice Dancing scores appeared to result in very little difference.

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

    File URL:
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.

    As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

    Article provided by De Gruyter in its journal Journal of Quantitative Analysis in Sports.

    Volume (Year): 7 (2011)
    Issue (Month): 3 (July)
    Pages: 1-23

    in new window

    Handle: RePEc:bpj:jqsprt:v:7:y:2011:i:3:n:16
    Contact details of provider: Web page:

    Order Information: Web:

    No references listed on IDEAS
    You can help add them by filling out this form.

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    When requesting a correction, please mention this item's handle: RePEc:bpj:jqsprt:v:7:y:2011:i:3:n:16. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Peter Golla)

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.