Challenges to evidence-based medicine: A comparison of patients and treatments in randomized controlled trials with patients and treatments in a practice research network

Background The practice of evidence-based medicine depends on the availability of clinically relevant research, yet questions have been raised about the generalizability of findings from randomized controlled trials (RCTs). Objective The aim of this study was to quantify differences between RCT patients and treatments and those in day-to-day clinical practice. Design Data from published reports of two key RCTs underlying recent treatment advances in psychiatry were compared with data on routine psychiatric practice collected through a Practice Research Network (PRN). Setting Hospital inpatient units (RCT) and the full range of psychiatric practice settings in the United States (PRN). Subjects Adults with bipolar I disorder and adults with schizophrenia. Measures Demographic (age, gender, race), clinical (principal diagnoses, comorbid conditions, psychosocial functioning, and histories of hospitalization), and treatment (medication name and dosage) characteristics. Results PRN patients had more comorbid conditions and were more likely to be white, female, and older than RCT patients. In all, 38% of PRN patients with schizophrenia and 55% of PRN patients with bipolar I disorder would have been ineligible for the corresponding RCT. Most PRN patients receiving an RCT study medication were also receiving other medications not allowed by the RCT protocol. Conclusions Findings support the assertion that RCT patients and treatments are not typical of those in clinical practice, and most patients in clinical practice are receiving treatments that do not have direct empirical support. Research is needed to determine the extent to which RCT findings should be used to guide routine clinical decisions.