The hazards of stroke case selection using administrative data

BACKGROUND. Administrative data and ICD-9-CM diagnostic codes are frequently used in research efforts to evaluate risk adjusted patient outcomes, particularly mortality. Varying ICD-9-CM sampling algorithms have been used to identify stroke patients. OBJECTIVES. This study evaluates the effects of different sampling strategies (one high sensitivity and one high specificity) on modeling stroke mortality as a performance indicator. RESEARCH DESIGN. Risk adjustment models were developed for two stroke cohorts identified using differing ICD-9-CM algorithms. Standard mortality ratios were calculated in a validation sample as network performance measures and compared across the two stroke samples. SUBJECTS. VHA inpatients with stroke during years 1997 (model development) and 1998 (model validation) were selected from the Patient Treatment File based on cerebrovascular diagnostic codes. MEASURES. Patient mortality within 30 days of admission. RESULTS. The model development and validation for each stroke sampling method produced consistent results: c-statistics 0.74 to 0.75, R-2 0.07 to 0.09, concordance 73% to 74%. However, ranking differences in network performance varied by 5 or more positions for 7 of the 22 patient networks. CONCLUSIONS. These findings highlight a potential problem when using administrative data to assess stroke mortality. In the absence of an agreed upon definition of stroke patients, results of provider profiling will vary depending on the ICD-9 algorithm used.