Comparing scaling methods for health state valuations - Rosser revisited

The methodology of valuing health states remains a key issue in the construction of health-related quality of life measures. Different scaling methods appear to yield different sets of valuations, and as yet there is no consensus as to which method is the preferred technique. Many of the cost-per-QALY estimates produced in the United Kingdom have been based on the Rosser Classification and its associated Valuation Matrix, which used the method of Magnitude Estimation. This paper reports on a study comparing the three main scaling methods (Category Rating, Magnitude Estimation and Time Trade-off) using the Rosser Classification to describe states of health. The objectives of the study were two-fold. First, to assess whether the Magnitude Estimation valuations obtained from a random sample of the general population correspond to the original values obtained from a convenience sample by Rosser, and second, to compare health state valuations produced by three different scaling methods. The values in the original Rosser matrix were not exactly reproduced in this study. Possible contributing factors are the different demographic characteristics of the respondents, as well as differences in the design of the study and in the detail of the Magnitude Estimation technique used. There is a high degree of consensus between Category Rating, Magnitude Estimation, and Time Trade-off methods in the ranking of states, however there appears to be important differences between the actual valuations produced by the different methods. In addition, valuations can be sensitive to order of presentation of method and interviewer bias. It appears from a reworking of cost-per-QALY estimates using these ‘new’ matrices that the precise “quality-adjustments” used in QALY estimates do matter. Given that no-one is yet in a position to claim to have established a “definitive” set of valuations, empirical cost-per-QALY measures this need to be interpreted with some caution. In the meantime, however, those wishing to calculate QALYs on the basis of Rosser’s descriptive system will have to choose from one of three alternatives – the original matrix, a new matrix based on Magnitude Estimation, and a ‘synthesised’ matrix based on a commonsense interpretation of scores from all three scaling methods. This may not be an easy choice, and for the time being it might be advisable to conduct analyses of QALY data using all three matrices so as to test the robustness of any conclusions to be drawn.