Measuring healthcare quality variation using multicategory ordinal data: an application to primary care services in England

The paper proposes a framework for measuring both the comparative quality of a set of healthcare providers and the variation in quality between them. The measures are directly calculable using the multicategory response data increasingly available from patient experience surveys but are also well defined for standard cardinal quality indicators. Moreover, they are sensitive to the full distribution of quality scores for each provider, not just the mean nor the proportion meeting some binary quality threshold. We illustrate our approach by providing comparable estimates of the variation in the quality of primary care services in England using three different sources of publicly available, general practice-level information: multicategory response patient experience data, ordinal inspection ratings and cardinal clinical achievement scores. Our results reveal considerable variation in the quality of primary care services at both local and regional level: for example, a randomly chosen patient from the best practice in England had a 26.0 percentage point higher chance of reporting a better rather than worse experience than a patient from anywhere in the country, whereas one from the worst practice had a 30.7 percentage point lower chance. Weak correlation between the comparative quality indices calculated using the alternative quality indicators provides evidence that inspection ratings and clinical process indicators capture different aspects of GP quality than patient experience. Finally we investigate the impact of standardisation, reporting results based on both raw and indirectly standardised practice quality profiles, with the latter based on the estimation of a distribution regression model.