Metacognitive efficiency in learned value-based choice

Metacognition, the ability to assess the quality of our own decisions, is a critical form of higher-order information processing. Metacognitive efficiency is therefore an essential measure of cognitive capability. When making decisions is simple, metacognitive judgments are also straightforward; thus, assessing metacognitive efficiency requires normalizing for the quality of underlying task performance. This is duly common in measures of efficiency popular in perceptual decision-making such as the M-ratio. However, such normalization is hard in reinforcement learning problems, because task difficulty changes dynamically. We therefore repurposed the central idea underlying the M-ratio, using confidence judgments to fashion a notional decision-maker (which we call a Backward model), assessing metacognitive sensitivity according to the quality of its virtual decisions, and quantifying metacognitive efficiency by comparing virtual and modelled actual qualities in the original decision-making task. We used simulated and empirical data to show that our measure of metacognitive sensitivity, the Backward performance, has comparable properties to other measures such as quadratic scoring, and that our measure of efficiency, the MetaRL.Ratio, is independent of empirical performance and is preserved across levels of task difficulty. We suggest that the MetaRL.Ratio as a promising tool for assessing metacognitive efficiency in value-based learning/decision-making.Author summary: When we make choices based on experience, we often judge how confident we are in them. The more accurate these so-called metacognitive judgments, the better, or more efficiently, we are able to evaluate our own decisions. Metacognitive efficiency plays an important role in many aspects of cognition and has been implicated in disease. In simple tasks such as spotting objects in a picture, many excellent ways have been developed to measure metacognitive efficiency. However, these methods struggle when the underlying difficulty of tasks fluctuates in incompletely known ways, as typically happens when people learn by trial and error. We introduce a new way to study metacognitive efficiency in such dynamic settings. We compare the synthetic performances of two models that can both solve the underlying task. A forward model is fit to match just the choices that participants made. A backward model is fit to match just the confidence judgements that accompanied those choices. The ratio between the two performances, the MetaRL.Ratio, is a measure of metacognitive efficiency that works well even as task difficulty shifts. In a game-like task in which reward values changed over time, the MetaRL.Ratio consistently measured metacognitive efficiency, unaffected by overall performance or confidence levels. Our measure helps extend the scope of metacognitive assessments.