Suprathreshold perceptual decisions constrain models of confidence

Perceptual confidence is an important internal signal about the certainty of our decisions and there is a substantial debate on how it is computed. We highlight three confidence metric types from the literature: observers either use 1) the full probability distribution to compute probability correct (Probability metrics), 2) point estimates from the perceptual decision process to estimate uncertainty (Evidence-Strength metrics), or 3) heuristic confidence from stimulus-based cues to uncertainty (Heuristic metrics). These metrics are rarely tested against one another, so we examined models of all three types on a suprathreshold spatial discrimination task. Observers were shown a cloud of dots sampled from a dot generating distribution and judged if the mean of the distribution was left or right of centre. In addition to varying the horizontal position of the mean, there were two sensory uncertainty manipulations: the number of dots sampled and the spread of the generating distribution. After every two perceptual decisions, observers made a confidence forced-choice judgement whether they were more confident in the first or second decision. Model results showed that the majority of observers were best-fit by either: 1) the Heuristic model, which used dot cloud position, spread, and number of dots as cues; or 2) an Evidence-Strength model, which computed the distance between the sensory measurement and discrimination criterion, scaled according to sensory uncertainty. An accidental repetition of some sessions also allowed for the measurement of confidence agreement for identical pairs of stimuli. This N-pass analysis revealed that human observers were more consistent than their best-fitting model would predict, indicating there are still aspects of confidence that are not captured by our modelling. As such, we propose confidence agreement as a useful technique for computational studies of confidence. Taken together, these findings highlight the idiosyncratic nature of confidence computations for complex decision contexts and the need to consider different potential metrics and transformations in the confidence computation.Author summary: The feeling of confidence in what we perceive can influence our future behaviour and learning. Understanding how the brain computes confidence is an important goal of researchers. As such, researchers have identified a host of potential models. Yet, rarely are a wide range of models tested against each other to find those that best predict choice behaviour. Our study had human participants compare their confidence for pairs of easy perceptual decisions, reporting if they had higher confidence in the first or second decision. We tested twelve models, covering all three types of models proposed in previous studies, finding strong support for two models. The winning Heuristic model combines all three factors affecting choice uncertainty with an idiosyncratic weighting to compute confidence. The other winning model uses a transformation where the strength of the sensory signal is scaled according to sensory uncertainty. We also assessed the agreement of confidence reports in identical decision scenarios. Humans had higher agreement than almost all model predictions. We propose using confidence agreement intentionally as a second performance benchmark of model fit.