A Comparison of Bayesian Methods for Profiling Hospital Performance

There is a growing interest in the use of Bayesian methods for profiling institutional performance. In the literature, several studies have compared different frequentist methods for classifying hospitals as performance outliers. The purpose of this study was to compare 4 different Bayesian methods for classifying hospitals as outcomes outliers, using 30-day hospital-level mortality rates for a cohort of acute myocardial infarction patients as a test case. The 1st Bayesian method involved determining the probability that a hospital’s mortality rate for an average patient exceeded a specified threshold. The 2nd method involved ranking hospitals according to their mortality rate for an average patient. The 3rd method involved determining the probability that a hospital’s standardized mortality ratio exceeded a specified threshold. The 4th method involved ranking hospitals according to their standardized mortality ratio. In most of the scenarios examined, there was only marginal agreement between the different methods. In only 4 of 19 comparisons, was there good agreement between the different methods (0.40 kappa 0.75). Methods based on ranking institutions were relatively insensitive to differences between hospitals. These inconsistencies raise questions about the choice of methods for classifying hospital performance, and they suggest a need for urgent research into which methods are best able to discriminate between institutions and which are most meaningful to decision makers.