Extreme Sports Evaluation: Evidence from Judging Figure Skating
AbstractThe quality of subjective performance evaluation is dependent on the incentive structures faced by evaluators, in particular on how they are monitored and themselves evaluated. Figure skating competitions provide a unique opportunity to study subjective evaluation. This paper develops a simple model of "outlier aversion bias" in which evaluators avoid submitting outlying judgments, because they are later assessed by the degree to which their scores match those of the other judges. I employ two empirical strategies: first, individual judges are traced, and within-individual dynamic correlations of scores are tested for by using the standard dynamic panel data model. Second, I exploit a natural experiment in which anonymity and random selection of judges have been introduced under new judging system in 2003. I find significant evidence for the existence of outlier aversion bias. Individual judges manipulate scores to achieve a targeted level of agreement with the other judges, and the dispersion of scores depends upon the type of judge-assessment system. One major implication of this paper, which is at variance with the industrial psychology and personnel management literature, is that agreement may not be a good criterion for the validity of an evaluation system. The price of agreement is at least a 14 percent loss of information
Download InfoTo our knowledge, this item is not available for download. To find whether it is available, there are three options:
1. Check below under "Related research" whether another version of this item is available online.
2. Check on the provider's web page whether it is in fact available.
3. Perform a search for a similarly titled item that would be available.
Bibliographic InfoPaper provided by Econometric Society in its series Econometric Society 2004 North American Summer Meetings with number 122.
Date of creation: 11 Aug 2004
Date of revision:
Contact details of provider:
Phone: 1 212 998 3820
Fax: 1 212 995 4487
Web page: http://www.econometricsociety.org/pastmeetings.asp
More information through EDIRC
Subjective Evaluation; Rational Herding;
Find related papers by JEL classification:
- D7 - Microeconomics - - Analysis of Collective Decision-Making
- M5 - Business Administration and Business Economics; Marketing; Accounting - - Personnel Economics
You can help add them by filling out this form.
reading lists or Wikipedia pages:
- Extreme sport in Wikipedia (English)
- رياضة عنيفة in Wikipedia (Arabic)
- Sport extrem in Wikipedia (Romanian)
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F. Baum).
If references are entirely missing, you can add them using this form.