A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury

From the beginning, the reporting of the results of National Acute Spinal Cord Injury Studies (NASCIS) II and III has been incomplete, leaving clinicians in the spinal cord injury (SCI) community to use or avoid using methylprednisolone in acute SCI on the basis of faith rather than a publicly developed scientific consensus. NASCIS II was initially reported by National Institutes of Health announcements, National Institutes of Health facsimiles to emergency room physicians, and the news media. The subsequent report in the New England Journal of Medicine implied that there was a positive result in the primary efficacy analysis for the entire 487 patient sample. However, this analysis was in fact negative, and the positive result was found only in a secondary analysis of the subgroup of patients who received treatment within 8 hours. In addition, that subgroup apparently had only 62 patients taking methylprednisolone and 67 receiving placebo. The NASCIS II and III reports embody specific choices of statistical methods that have strongly shaped the reporting of results but have not been adequately challenged or or even explained. These studies show statistical artifacts that call their results into question. In NASCIS II, the placebo group treated before 8 hours did poorly, not only when compared with the methylprednisolone group treated before 8 hours but even when compared with the placebo group treated after 8 hours. Thus, the positive result may have been caused by a weakness in the control group rather than any strength of methylprednisolone. In NASCIS III, a randomization imbalance occurred that allocated a disproportionate number of patients with no motor deficit (and therefore no chance for recovery) to the lower dose control group. When this imbalance is controlled for, much of the superiority of the higher dose group seems to disappear. The NASCIS group's decision to admit persons with minor SCIs with minimal or no motor deficit not only enables statistical artifacts it complicates the interpretation of results from the population actually sampled. Perhaps one half of the NASCIS III sample may have had at most a minor deficit. Thus, we do not know whether the results of these studies reflect the severely injured population to which they have been applied. The numbers, tables, and figures in the published reports are scant and are inconsistently defined, making it impossible even for professional statisticians to duplicate the analyses, to guess the effect of changes in assumptions, or to supply the missing parts of the picture. Nonetheless, even 9 years after NASCIS II, the primary data have not been made public. The reporting of the NASCIS studies has fallen far short of the guidelines of the ICH/FDA and of the Evidence-based Medicine Group. Despite the lucrative "off label" markets for methylprednisolone in SCI, no Food and Drug Association indication has been obtained. There has been no public process of validation. These shortcomings have denied physicians the chance to use confidently a drug that many were enthusiastic about and has left them in an intolerably ambiguous position in their therapeutic choices, in their legal exposure, and in their ability to perform further research to help their patients.