Measuring resource use in the ICU with computerized therapeutic intervention scoring system-based data

Background and objective: In this era of health-care reform, there is increasing need to monitor and control health-care resource consumption. This requires the development of measurement tools that are practical, uniform, reproducible, and of sufficient detail to allow comparison among institutions, among select groups of patients, and among individual patients. We explored the feasibility of generating and index of resource use based on the Therapeutic Intervention Scoring System (TISS) from hospital electronic billing data. Such an index is potentially comparable across institutions, allows assessment of care at many levels, is well understood by clinicians, and captures many of the resources relevant to the ICU. Design: We develop an automated mapping of the hospital billing database into different items of TISS and generated computerized active TISS scores on 1,372 ICU days. The computerized score was then validated by comparison to prospectively gathered active TISS scores by trained data collectors. Setting: Eight ICUs within a university teaching institution. Patients: We studied 1,229 general medical and surgical ICU patients. Interventions: None. Measurements and main results: Active TISS scores ranged from 0 to 31 points. The two scores were well correlated (R-2=0.53) and highly calibrated (as assessed by regression of active TISS on mean computerized active TISS [R-2=0.85]). The scores were identical on 756 days (55.6%) and differed by less than or equal to 3 TISS points on an additional 387 (28.2%) days. Interreliability assessment suggested substantial agreement (kappa statistic=0.71). The discriminatory power of the computerized score to identify different levels of ICU resource use was excellent as assessed by area under the receiver operating characteristics curves at four threshold points (0.91, 0.87, 0.89, and 0.88). Performance of the computerized score was similar across medical, coronary, and surgical ICU patient groups. Conclusion: An automated algorithm can reproduce valid TISS scores from standard hospital billing data, allowing comparison of patients and groups of patients in order to better understand ICU resource use.