How improper dichotomization and the misrepresentation of uncertainty undermine social science research

If the fundamental problem in social science research was a matter of researchers “gaming” their p-values or even unintentionally accumulating “significant findings,” then steps such as monitoring the distribution of p-values in research could help to resolve the problem. Yet “significant” p-values do not indicate a “finding” nor their absence a “non-finding,” so these efforts may be misguided. More fundamental problems lie in researchers’ false dichotomization of intrinsically continuous p-values and in researchers’ misrepresentation of the uncertainty of their findings. Statistical standard errors as conventionally computed in social science research substantially understate the overall uncertainty of research results. Disregarding the meaningless concept of “statistical significance” and adopting a metrological view of uncertainty will better enable researchers, journals and disciplines to grasp the scientific contributions of individual research studies and to design future studies that will make a genuine contribution to the state of knowledge.