Using artificial neural networks to reveal the human confidence computation

Humans can evaluate the accuracy of their decisions by providing confidence judgements. Traditional cognitive models have tried to capture the mechanisms underlying this process but remain mostly limited to two-choice tasks with simple stimuli. How confidence is given for naturalistic stimuli with multiple choice alternatives is not well understood. We recently developed a convolutional neural network (CNN) model – RTNet – that exhibits several important signatures of human decision making and outperforms existing alternatives in predicting human responses. Here, we use RTNet’s image computability to test seven different confidence strategies on a task involving digit discrimination with eight alternatives (N = 60). Specifically, we compared confidence strategies that consider the entire evidence distribution across choices against strategies that only consider a specific subset of the available evidence. We also compared strategies based on whether they compute confidence from posterior probabilities or from raw evidence. A model which derives confidence from the difference in raw evidence between the top-two choices (Top2Diff model) consistently provided the best quantitative and qualitative fits to the data and the best predictions of human confidence. These results support the notion that human confidence is based on a specific subset of available evidence, challenge prominent theories such as the Bayesian confidence hypothesis and the positive evidence heuristic, and establish CNNs as promising models for inferring the mechanisms underlying human confidence in naturalistic settings.Author summary: Human decisions are accompanied by a sense of confidence which reflects the accuracy of their decisions. Conventionally, human confidence has been studied using two-choice tasks with simple stimuli such as Gabor patches and dots, making it unclear how confidence is given for naturalistic stimuli with several choice alternatives. In this study, we used a neural network model called RTNet to investigate human confidence mechanisms in an eight-choice task with handwritten digits. We instantiated seven confidence strategies that differed based on: 1) how much perceptual evidence is incorporated into confidence, and 2) whether confidence uses the raw perceptual evidence or computes probabilities from this evidence. Our results showed that human confidence was best explained by a model that derives confidence from the difference in raw evidence between the top-two choices. These results support the notion that human confidence is based on a specific subset of the raw evidence and establish a framework for inferring the mechanisms underlying human confidence in naturalistic settings.