Individual differences in the perception of probability

In recent studies of humans estimating non-stationary probabilities, estimates appear to be unbiased on average, across the full range of probability values to be estimated. This finding is surprising given that experiments measuring probability estimation in other contexts have often identified conservatism: individuals tend to overestimate low probability events and underestimate high probability events. In other contexts, repulsive biases have also been documented, with individuals producing judgments that tend toward extreme values instead. Using extensive data from a probability estimation task that produces unbiased performance on average, we find substantial biases at the individual level; we document the coexistence of both conservative and repulsive biases in the same experimental context. Individual biases persist despite extensive experience with the task, and are also correlated with other behavioral differences, such as individual variation in response speed and adjustment rates. We conclude that the rich computational demands of our task give rise to a variety of behavioral patterns, and that the apparent unbiasedness of the pooled data is an artifact of the aggregation of heterogeneous biases.Author summary: Humans often misrepresent probabilities, frequencies, and proportions they encounter—either overestimating or underestimating the true underlying values. Understanding the cognitive and neural representation of such quantities is important, as probabilities are present in all kinds of impactful decisions—e.g., as we assess the likelihood of a dangerous event or the probable returns on a financial investment. Despite the ubiquitous observation that humans are biased in estimating probabilities, a new laboratory task reportedly elicits unbiased performance. This task involves predicting the likelihood of a green ring emerging from a box containing red and green rings; as subjects draw rings and report their estimates, the ring composition of the box itself changes infrequently. Here we perform a novel and thorough analysis of individuals’ performance on one such experiment, showing that people are indeed biased in idiosyncratic ways. These patterns are unexamined by studies focusing on average performance: some people routinely report an exaggerated range of probabilities, some favor intermediate values, while others are approximately unbiased. These biases persist across hours of experience and accompany other tendencies such as quickness in response. We show that subjects’ idiosyncrasies can be understood as arising from various cognitive mechanisms; nonetheless, subjects reports are best described as resembling a distorted version of the optimal statistical estimate.