Discovering treatment effectiveness via median treatment effects—Applications to COVID‐19 clinical trials

Comparing median outcomes to gauge treatment effectiveness is widespread practice in clinical and other investigations. While common, such difference‐in‐median characterizations of effectiveness are but one way to summarize how outcome distributions compare. This paper explores properties of median treatment effects (TEs) as indicators of treatment effectiveness. The paper's main focus is on decisionmaking based on median TEs and it proceeds by considering two paths a decisionmaker might follow. Along one, decisions are based on point‐identified differences in medians alongside partially identified median differences; along the other decisions are based on point‐identified differences in medians in conjunction with other point‐identified parameters. On both paths familiar difference‐in‐median measures play some role yet in both the traditional standards are augmented with information that will often be relevant in assessing treatments' effectiveness. Implementing either approach is straightforward. In addition to its analytical results the paper considers several policy contexts in which such considerations arise. While the paper is framed by recently reported findings on treatments for COVID‐19 and uses several such studies to explore empirically some properties of median‐treatment‐effect measures of effectiveness, its results should be broadly applicable.