Validating a satisfaction questionnaire using multiple approaches: A case study

We examined the validity of a questionnaire designed to measure the satisfaction of users of health services, using multiple tests of construct validity. Members of 2 health insurance plans in Geneva (Switzerland) answered a mailed questionnaire in 1992 (n = 1007) and 1993 (n = 1424). Response rates were 82%, participants were 18-44 years old in 1992. The questionnaire included 22 questions on satisfaction with medical care received during the past 12 months. Most items were adapted from the Patient Satisfaction Questionnaire. Four dimensions of satisfaction were measured: satisfaction with physician services (8 items), communication (8 items), access (4 items) and insurance services (2 items). Reliability (Cronbach's [alpha]) was satisfactory for the 2 former dimensions ([alpha] = 0.81 and 0.82 respectively), but lower than desired for the 2 latter ([alpha] = 0.63 and 0.49 respectively). Participants who gave positive open-ended comments had satisfaction scores 0.7-1.2 standard deviation units higher than participants who gave negative comments. Satisfaction scores were weakly correlated with satisfaction with private life, which indicates that the instrument did not simply measure a general tendency to be satisfied. Participants who said that care received in 1993 was worse than care received in 1992 (retrospective assessment) experienced a decrease in satisfaction scores between 1992 and 1993 (prospective assessment). Most validation procedures provided independent but partial evidence for the validity of the instrument. Triangulation of several validation methods, as illustrated in this paper, may greatly improve the understanding of an instrument's properties.