The Good, the Bad, and the Bayesian: Self-Confidence and Algorithmic Advice

Understanding how decision makers incorporate external information is increasingly important as algorithmic predictions become a common source of advice. The literature on advice-taking suggests that people discount others' advice, especially when they feel competent. We experimentally study how self-confidence affects advice utilization and how algorithmic advice shapes confidence in final judgments. In our online experiment, participants make age predictions and assess the prediction accuracy before and after receiving algorithmic advice. We exogenously vary self-confidence through (perceived) task difficulty and vary the (observable) quality of the algorithmic advisor. While participants generally put more weight on high-quality ("good") advice, we find, contrary to the existing advice-taking literature, that they are less influenced by advice when they perceive the task as difficult. By contrasting age and accuracy predictions, we show that advice can influence beliefs without substantially changing point predictions: people who feel proficient at a task barely adjust their predictions toward high-quality advice but become more confident in them. When people feel less proficient, they are more reluctant to adjust their predictions toward low-quality ("bad") advice. Our design shows that prediction and confidence updating respond differently to perceived task difficulty and advice quality, implying that measures based solely on changes in point predictions can misstate the influence of advice. Comparing observed updating with a Bayesian benchmark further reveals systematic underutilization of advice, driven in particular by insufficient confidence updating following high-quality advice in tasks perceived as easy.