The discriminative power of patient experience surveys

Background: Comparisons of patient experiences between providers are increasingly used as an index of performance. The present study describes the ability of patient experience surveys to discriminate between healthcare providers for various patient groups and quality aspects, and reports the sample sizes required for reliable (comparisons of) provider scores. Method: The consumer quality index is a family of surveys that are tailored to specific patient groups. Data was used from patients who underwent cataract surgery, patients who underwent hip or knee surgery, patients suffering from spinal disc herniation and patients suffering from varicose veins. Multi-level regression models were fitted to assess the proportion of variance in patient experiences that is attributable to providers for various quality aspects. Results: The proportion of variance in patient experiences that is attributable to providers varied from 0.001 to 0.054. The required sample size for reliable estimates at the provider level varied from 41 to 1967 per provider. Differences in discriminative power between patient groups and/or quality aspects were inconsistent, with one exception: for all groups, the discriminative power of experiences regarding change in physical functioning was particularly limited. Conclusions: From a statistical point of view, the discriminative power appears limited. The sample sizes required for reliable estimates are often substantial and deserve careful consideration when setting up measurements. Future research should evaluate the discriminative power by validating differences between providers in patient experiences with other indices and should explore other, more sensitive measures of patient experiences regarding treatment-related changes in physical functioning.