Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups

Objective To examine the efficacy of different methods of detecting a high death rate and determining whether an increase in deaths after heart transplantation could be explained by chance. Design Retrospective analysis of deaths after heart transplantation. Seven methods were used: mortality above national average, mortality excessively above national average, test of moving average mortality, test of number of consecutive deaths, sequential probability ratio test (SPRT), cusum with v-mask, and CRAM chart. The national average mortality was not available, and a rate of 15% was used instead as the benchmark. Setting Regional cardiothoracic unit. Participants All 371 patients who received a heart transplant in the programme, 1986-2000. Main outcome measures 30 day survival after transplantation. Results All methods provided evidence that the 30 day mortality had been high at some stage. The probability that the finding was a false positive depended on which test was used. At the end of the series the average mortality, sequential probability ratio, and cusum tests indicated a level of deaths higher than the benchmark while the remaining four tests yielded negative results. Conclusions If the decision to test for outlying mortality is made retrospectively, in the light of the data, it is not possible to determine the false positive rate. Prospective on-site mortality monitoring with the CRAM chart is recommended as this method can quantify the death rate and identify periods when an audit of cases is indicated, even when data from other hospitals are not available. A hospital mortality monitoring group can routinely monitor all deaths in the hospital, by specialty, using hospital episode statistics (HES) data and appropriate statistical methods.