A brief primer on automated signal detection

BACKGROUND: Statistical techniques have traditionally been underused in spontaneous reporting systems used for postmarketing surveillance of adverse drug events. Regulatory agencies, pharmaceutical companies, and drug monitoring centers have recently devoted considerable efforts to develop and implement computer-assisted automated signal detection methodologies that employ statistical theory to enhance screening efforts of expert clinical reviewers. OBJECTIVE: To provide a concise state-of-the-art review of the most commonly used automated signal detection procedures, including the underlying statistical concepts, performance characteristics, and outstanding limitations, and issues to be resolved. DATA SOURCES: Primary articles were identified by MEDLINE search (1965-December 2002) and through secondary sources. STUDY SELECTION AND DATA EXTRACTION: All of the articles identified from the data sources were evaluated and all information deemed relevant was included in this review. DATA SYNTHESIS: Commonly used methods of automated signal detection are self-contained and involve screening large databases of spontaneous adverse event reports in search of interestingly large disproportionalities or dependencies between significant variables, usually single drug-event pairs, based on an underlying model of statistical independence. The models vary according to the underlying model of statistical independence and whether additional mathematical modeling using Bayesian analysis is applied to the crude measures of disproportionality. There are many potential advantages and disadvantages of these methods, as well as significant unresolved issues related to the application of these techniques, including lack of comprehensive head-to-head comparisons in a single large transnational database, lack of prospective evaluations, and the lack of gold standard of signal detection. CONCLUSIONS: Current methods of automated signal detection are nonclinical and only highlight deviations from independence without explaining whether these deviations are due to a causal linkage or numerous potential confounders. They therefore cannot replace expert clinical reviewers, but can help them to focus attention when confronted with the difficult task of screening huge numbers of drug-event combinations for potential signals. Important questions remain to be answered about the performance characteristics of these methods. Pharmacovigilance professionals should take the time to learn the underlying mathematical concepts in order to critically evaluate accumulating experience pertaining to the relative performance characteristics of these methods that are incompletely defined.