Validation of "nine equivalents of nursing manpower use score" on an independent data sample

Objective: To compare the recently developed "nine equivalents of nursing manpower use score" (NEMS) with the simplified Therapeutic Intervention Scoring System (TISS-28). Design: Prospective single centre study. Setting: Adult 30-bed medical-surgical intensive care unit (ICU) in a tertiary care university hospital. Patients: Data from all patients admitted in 1997 to the ICU were included in the study. Methods and results: NEMS and TISS-28 items were recorded prospectively for each nursing shift. There were three shifts per day. The Simplified Acute Physiology Score (SAPS) II was calculated for the first 24 h of ICU stay and each patient's basic demographic data were collected. The agreement between NEMS and TISS-28 was assessed by calculating the mean difference and the standard deviation of the differences between the two measures. Further, regression techniques and Pearson's correlation were used. Altogether, 2743 patients with a total of 28'220 nursing shifts were included: 62 % of the shifts were used for postoperative/trauma patients and 38 % for medical patients, Mean NEMS was 26.0 +/- 8.1 and mean TISS-28 was 26.5 +/- 7.9. The scores differed by less than or equal to 3 points in 49 % of all shifts. The bias was -0.5 +/- 5.3 (95% confidence interval -0.47 to -0.60) and the limits of agreement were -11.1 to +10.1. The relation between the two systems was NEMS = 4.7 +/- 0.8 TISS-28 (r = 0.78, r(2) = 0.62, p < 0.001). Including postoperative/trauma patients only: NEMS = 1.9 + 0.9 TISS-28, for medical patients this equation was: NEMS = 6.0 + 0.8 TISS-28. First-day SAPS II explained 11 % of the variability in first-shift NEMS and 5% of the variability in first-shift TISS-28. Conclusions: This study confirms a good agreement between TISS-28 and NEMS in a large, independent sample. However, as shown by the differences between medical and postoperative/trauma patients, a change in case mix may result in different regression equations. Further, wide limits of agreement indicate that there may be a rather large variability between the two measures at the individual level.