Reliability and agreement in technical and artistic scores during real-time judging in two European acrobatic gymnastic events

The study investigates reliability measures for performance scores given by random judges (Execution and Artistic scores) and the agreement between more expert judges (Control Scores) and the two middle Execution/Artistic scores (Panel scores) during Qualifying and Final competitions performed at the 10th European Age Group Competition (EAGC) and at the 29th European Acrobatic Gymnastics Championships (ECh). To assess agreement, mean absolute Control-Panel score deviations were calculated using Bland-Altman and Kaplan-Meier plots. Reliability average measures ranged from 0.88 to 0.97. Control Score reliability was higher in ECh for Execution and Artistic scores. Artistic scores showed higher variability and lower agreement between judges in Qualification for ECh. Significant differences were found in Control and Panel score mean deviations for Execution scores at the EAGC and for Artistic scores at the ECh. No significant reliability differences were found between competitions. Control-and-Panel score correlations ranged from 0.862 to 0.915 for Execution scores and from 0.864 to 0.899 for Artistic scores. Overall results exceeded 80% agreement for Execution and Artistic scores. Reliability and strength of agreement of judging were high and very acceptable respectively. We suggest that judging reliability should be regularly examined to optimise technical and artistic criteria set by FIG for fairer competition.