The high degree of consistency in our results, across topics, magnitudes of uncertainty, and communication formats suggest that people “can handle the truth.” [1]
Uncertainty affects practically all important medical decisions. Therefore, effective decision aiding methods need to help people manage the uncertainties inherent in decisions they need to make.
There are two types of uncertainty that affect decision making. The first, called aleatory, is an uncertainty that we have no way of knowing in advance, such as the weather next year on Halloween. The second, called epistemic, refers to uncertainties due to the limitations of current knowledge that could be clarified or eliminated through further study.
In the October 28, 2022 Musing, I discussed a paper titled “Current best practice for presenting probabilities in patient decision aids: Fundamental principles” by Carissa Bonner and colleagues. [2] In it, they advised decision creators to “be cautious” about communicating epistemic uncertainties regarding the data included in the aid:
Be cautious about communicating second-order, epistemic uncertainty (e.g., using probability ranges), given that this uncertainty may be psychologically aversive and difficult to understand, and that optimal methods of communication remain to be determined” (Bonner et al., 2021, p. 824)
In contrast to concerns about the potential adverse effects of communicating uncertainties, there are good arguments supporting the need for communicating scientific information accurately and transparently, particularly when the communication is designed to support good decision making.
In 2020, Anne Marthe van der Bles and colleagues published an open access paper in the Proceedings of the National Academy of Sciences describing a series of experiments designed to assess the effects of communicating epistemic uncertainty, quantitatively and using verbal expressions, on people’s cognitions, emotions, and trust. [1] They found little evidence to support withholding information about evidence uncertainty. This is how they summarize their findings:
Overall, we found little evidence to suggest that communicating numerical uncertainty about measurable facts and numbers backfires or elicits psychological reactance. Across five high powered studies and an internal metaanalysis, we show that people do recognize and perceive uncertainty when communicated around point estimates, both verbally and numerically (except when only words such as “estimated” or “about” are used to imply uncertainty). In addition, uncertainty did not seem to influence their affective reaction … and although the provision of uncertainty in general did slightly decrease people’s trust in and perceived reliability of the numbers, this effect emerged for explicit verbal uncertainty in particular.
Musings
Although only one paper, the five studies included in the report by van der Bles and colleagues strongly suggests that gains in insight and transparency are likely to far outweigh the adverse effects of including quantitative information about the uncertainty of data included in decision aids. In addition to the outcomes measured in the study, including uncertainty information will also help people avoid the “flaw of averages”. This concept, developed by Sam Savage, simply states that decisions based on an average value frequently fail because the underlying variation in the quantity being measured is not taken into account. [3]
The key task for decision aid creators, and all those tasked with helping others make good decisions, is to help people effectively manage the uncertainties inherent in the data being considered. One way is to simply present the data graphically. A useful glossary of methods for displaying data distributions is on the Tableau Public website. Additional well-established methods for helping decision makers understand and manage future uncertainties are Monte Carlo simulations and scenario analysis. Many financial institutions use Monte Carlo simulations to project retirement income and savings. Sam Savage and colleagues have constructed an ingenious method of embedding Monte Carlo simulations in individual cells of spreadsheets that could be easily integrated into a medical decision aid that I think merits further investigation. [4]
References
1. van der Bles AM, van der Linden S, Freeman ALJ, Spiegelhalter DJ. The effects of communicating uncertainty on public trust in facts and numbers. Proceedings of the National Academy of Sciences. 2020 Apr 7;117(14):7672–83.
2. Bonner, Carissa et al. 2021. “Current Best Practice for Presenting Probabilities in Patient Decision Aids: Fundamental Principles.” Medical Decision Making 41(7): 821–33.
3. Savage S. The Flaw of Averages. Harvard Business Review [Internet]. 2002 Nov 1 [cited 2023 Mar 15]; Available from: https://hbr.org/2002/11/the-flaw-of-averages
4. Probability Management [Internet]. Probability Management. 2023 [cited 2023 Mar 15]. Available from: https://www.probabilitymanagement.org