Dissecting Bayes: Using influence measures to test normative use of probability density information derived from a sample

Bayesian decision theory (BDT) is frequently used to model normative performance in perceptual, motor, and cognitive decision tasks where the possible outcomes of actions are associated with rewards or penalties. The resulting normative models specify how decision makers should encode and combine information about uncertainty and value–step by step–in order to maximize their expected reward. When prior, likelihood, and posterior are probabilities, the Bayesian computation requires only simple arithmetic operations: addition, etc. We focus on visual cognitive tasks where Bayesian computations are carried out not on probabilities but on (1) probability density functions and (2) these probability density functions are derived from samples. We break the BDT model into a series of computations and test human ability to carry out each of these computations in isolation. We test three necessary properties of normative use of pdf information derived from a sample–accuracy, additivity and influence. Influence measures allow us to assess how much weight each point in the sample is assigned in making decisions and allow us to compare normative use (weighting) of samples to actual, point by point. We find that human decision makers violate accuracy and additivity systematically but that the cost of failure in accuracy or additivity would be minor in common decision tasks. However, a comparison of measured influence for each sample point with normative influence measures demonstrates that the individual’s use of sample information is markedly different from the predictions of BDT. We will show that the normative BDT model takes into account the geometric symmetries of the pdf while the human decision maker does not. An alternative model basing decisions on a single extreme sample point provided a better account for participants’ data than the normative BDT model.Author summary: Bayesian decision theory (BDT) is used to model human performance in tasks where the decision maker must compensate for uncertainty in order to gain rewards and avoid losses. BDT prescribes how the decision maker can combine available data, prior knowledge, and value to reach a decision maximizing expected winnings. Do human decision makers actually use BDT in making decisions? Researchers typically compare overall human performance (total winnings or overall percent correct) to the predictions of BDT but we cannot conclude that BDT is an adequate model for human performance based on just overall performance. We break BDT down into elementary operations and test human ability to execute such operations. In two of the tests human performance deviated only slightly (but systematically) from the predictions of BDT. In the third test, we use a novel method to measure the influence of each sample point provided to the human decision maker and compare it to the influence predicted by BDT. When we look at what human decision makers do–in detail–we find that they use sensory information very differently from what the normative BDT decision maker does. We advance an alternative non-Bayesian model that better predicts human performance. We propose that influence measures are a more sensitive way to discover discrepancies between human and optimal performance, than comparing overall performance.