A possible disconnect in JDM research articles: Results sections display an estimation perspective while Discussion sections belie an estimation mindset

Many journals ask authors to report confidence intervals (to quantify estimation precision or uncertainty) and measures of effect size (to quantify a factor’s explanatory power). Arguments for such practices focus on benefits to interpreting and applying scientific findings that go beyond merely detecting effects, thereby implying that effect sizes and confidence intervals should be reported and discussed. Accordingly, we examined 150 recent articles from 6 journals that publish research on Judgment and Decision Making (JDM) to survey current practices for reporting and discussing results. We recorded which of those articles report p-values, standardized effect sizes, and confidence/credibility intervals in their Results sections. We examined the articles’ narrative sections (Abstract, Discussion/Conclusion) for explicit reference to the presence/absence of an effect, an effect’s size, and the precision or range associated with an estimate. Ninety-one percent of articles reported p-values, CI0.95 [85%, 95%], and all discussed the presence or absence of effects. Most articles gave effect size information, with 73%, CI0.95 [65%, 79%], reporting standardized effect sizes, and 63%, CI0.95 [55%, 71%], reporting confidence/credibility intervals or graphical SE bars. However, an estimation perspective was less apparent in the articles’ Discussion sections, wherein 59%, CI0.95 [51%, 66%], discussed effect size information—though often with limited detail—and only 3%, CI0.95 [1%, 6%], discussed interval estimates. Mostly, it seems, JDM researchers follow guidelines for reporting effect size and the uncertainty and precision for effect estimates. Yet, one might ask whether this impacts researchers’ interpretation and communication of those effects as it should.