Problems arising from the generalising of treatment efficacy from clinical trials in Alzheimer's disease - Mistaking statistical significance as clinical significance

Objective: To determine the accuracy a practicing physician could expect in assessments of drug response in Alzheimer's disease (AD). Methods: We used the Mini-Mental State Examination as a benchmark for the practising physician's ability to quantify cognitive capacity in Alzheimer's disease without referral for assessment by a neuropsychologist. We derived effect sizes for treatment and confidence intervals (CIs) for this and another method of assessment [Alzheimer Disease Assessment Scale-Cognitive subscale (ADAS-Cog)] from their use in two recent clinical trials of metrifonate for AD. Results: CIs for the benchmark scale outcomes did not distinguish, with a known level of probability of error in judgment (5%), a drug response for a majority of individual patients from variation in test-retest scores from other causes. Conclusions: Our inability to distinguish the responses of individual patients from chance variations in retest scores using the best methods of assessment available to a clinician in practice precludes clinicians from reaching meaningful assessments of most AD patients' responses to the currently available treatments. Under these limitations clinicians require full disclosure of results from all clinical trials to reach informed decisions about patient care. Current practices of reporting clinical trials without providing appropriate measures of treatment effect size and CIs for outcome measures do not allow readers to assess the magnitude of the drug effect being reported or the probability of a detectable effect occurring in individual patients in the clinical setting. Thus informed decisions about the significance and method of use of drug treatments in the clinic are unable to be made.