IDEAS home Printed from
   My bibliography  Save this paper

Extreme Sports Evaluation: Evidence from Judging Figure Skating


  • Jungmin Lee


The quality of subjective performance evaluation is dependent on the incentive structures faced by evaluators, in particular on how they are monitored and themselves evaluated. Figure skating competitions provide a unique opportunity to study subjective evaluation. This paper develops a simple model of "outlier aversion bias" in which evaluators avoid submitting outlying judgments, because they are later assessed by the degree to which their scores match those of the other judges. I employ two empirical strategies: first, individual judges are traced, and within-individual dynamic correlations of scores are tested for by using the standard dynamic panel data model. Second, I exploit a natural experiment in which anonymity and random selection of judges have been introduced under new judging system in 2003. I find significant evidence for the existence of outlier aversion bias. Individual judges manipulate scores to achieve a targeted level of agreement with the other judges, and the dispersion of scores depends upon the type of judge-assessment system. One major implication of this paper, which is at variance with the industrial psychology and personnel management literature, is that agreement may not be a good criterion for the validity of an evaluation system. The price of agreement is at least a 14 percent loss of information

Suggested Citation

  • Jungmin Lee, 2004. "Extreme Sports Evaluation: Evidence from Judging Figure Skating," Econometric Society 2004 North American Summer Meetings 122, Econometric Society.
  • Handle: RePEc:ecm:nasm04:122

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    More about this item


    Subjective Evaluation; Rational Herding;

    JEL classification:

    • D7 - Microeconomics - - Analysis of Collective Decision-Making
    • M5 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics


    This item is featured on the following reading lists or Wikipedia pages:
    1. Sport extrem in Wikipedia Romanian ne '')
    2. رياضة عنيفة in Wikipedia Arabic ne '')


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecm:nasm04:122. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F. Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.