Evidence on Methods for Communicating Health-Related Probabilities: Comparing the Making Numbers Meaningful Systematic Review to the 2021 IPDAS Evidence Paper Recommendations

Purpose To summarize the degree to which evidence from our recent Making Numbers Meaningful (MNM) systematic review of the effects of data presentation format on communication of health numbers supports recommendations from the 2021 International Patient Decision Aids Standards (IPDAS) Collaboration papers on presenting probabilities. Methods The MNM review generated 1,119 distinct findings (derived from 316 papers) related to communication of probabilities to patients or other lay audiences, classifying each finding by its relation to audience task, type of stimulus (data and data presentation format), and up to 10 distinct sets of outcomes: identification and/or recall, contrast, categorization, computation, probability perceptions and/or feelings, effectiveness perceptions and/or feelings, behavioral intentions or behavior, trust, preference, and discrimination. Here, we summarize the findings related to each of the 35 IPDAS paper recommendations. Results Strong evidence exists to support several IPDAS recommendations, including those related to the use of part-to-whole graphical formats (e.g., icon arrays) and avoidance of verbal probability terms, 1-in-X formats, and relative risk formats to prevent amplification of probability perceptions, effectiveness perceptions, and/or behavioral intentions as well as the use of consistent denominators to improve computation outcomes. However, the evidence base appears weaker and less complete for other IPDAS recommendations (e.g., recommendations regarding numerical estimates in context and evaluative labels). The IPDAS papers and the MNM review agree that both communication of uncertainty and use of interactive formats need further research. Conclusions The idea that no one visual or numerical format is optimal for every probability communication situation is both an IPDAS panel recommendation and foundational to the MNM project’s design. Although no MNM evidence contradicts IPDAS recommendations, the evidence base needed to support many common probability communication recommendations remains incomplete. Highlights The Making Numbers Meaningful (MNM) systematic review of the literature on communicating health numbers provides mixed support for the recommendations of the 2021 International Patient Decision Aids Standards (IPDAS) evidence papers on presenting probabilities in patient decision aids. Both the IPDAS papers and the MNM project agree that no single visual or numerical format is optimal for every probability communication situation. The MNM review provides strong evidentiary support for IPDAS recommendations in favor of using part-to-whole graphical formats (e.g., icon arrays) and consistent denominators. The MNM review also supports the IPDAS cautions against verbal probability terms and 1-in-X formats as well as its concerns about the potential biasing effects of relative risk formats and framing. MNM evidence is weaker related to IPDAS recommendations about placing numerical estimates in context and use of evaluative labels.