Natural Frequencies Improve Public Understanding of Medical Test Results: An Experimental Study on Various Bayesian Inference Tasks with Multiple Scoring Methods and Non-Bayesian Reasoning Strategies

Background It is well established that the natural frequencies (NF) format is cognitively more beneficial for Bayesian inference than the conditional probabilities (CP) format. However, empirical studies have suggested that the NF facilitation effect might be limited to specific groups of individuals. Unlike previous studies that focused on a limited number of Bayesian inference problems evaluated by a single scoring method, it was essential to examine multiple Bayesian problems using various scoring metrics. This study also explored the impact of numeracy on Bayesian inference and assessed non-Bayesian cognitive strategies using the numerical information in problem solving. Methods In a Web-based experimental survey, 175 South Korean adults were randomly assigned to 1 of 2 format groups (NF v. CP). After completing numeracy scales, participants were asked to estimate 4 Bayesian inference problems and document the numerical information used in their problem-solving process. Four scoring methods—strict rounding, loose rounding, absolute deviation, and 50-Split—were used to evaluate participants’ estimations. Results The NF format generally outperformed the CP format across all problems, except in a chorionic villus sampling test problem when evaluated using the 50-Split method. In addition, numeracy levels significantly influenced Bayesian inference; participants with higher numeracy demonstrated better performance. In addition, participants used various non-Bayesian strategies influenced by the format and the nature of the problems. Conclusions The NF facilitation effect was consistently observed across multiple Bayesian problems and scoring methods. Individuals with higher numeracy levels benefited more from the NF format. The use of various non-Bayesian strategies varied with the formats and nature of specific tasks. Highlights The natural frequencies (NF) format is known to foster understanding of medical test results compared with the conditional probabilities (CP) format, but some studies have reported that this benefit is either nonexistent or limited to specific groups. This study aims to replicate previous empirical studies using various Bayesian problems using multiple scoring methods. The NF format fosters understanding of medical test results across all Bayesian problems by all scoring methods, except in the CVS problem when using a 50-Split scoring method. Participants with high numeracy perform better Bayesian inference than those with lower numeracy. Particularly, higher numerates benefit more in the NF format than lower numerates do. In addition, the public tend to use various non-Bayesian reasoning strategies depending on the format and the nature of the tasks.