On the triangle test with replication

We consider the triangle test with replications, i.e. each assessor is asked repeatedly. A commonly used test statistic for this situation is the sum of all correct assessments, summed over all assessors. Several authors (e.g. o’Mahony, 1982, Brockhoff and Schlich, 1998) argue that the binomial distribution cannot be used to analyse this kind of data. Brockhoff and Schlich (1998) propose an alternative model for the triangular test with replicates, where the assessors have different probabilities to correctly identify the odd sample even if the products are identical. Although we agree that assessors will have different probabilities of correct assessment if there are true differences, we do not think that Brockhoff and Schlich’s model makes sense under the null hypothesis of equality of treatments. We show that all assessments are independent and have success probability 1/3, if the null hypothesis is true and the experiment is properly randomized and properly carried out. This implies that the sum of all correct assessments is binomial with parameter p = 1/3. Therefore the usual test based on this sum and the critical values of the binomial distribution is a level α test for the null hypothesis of equality of the products, even if there are replications.