Interrater reliability among epilepsy centers: Multicenter study of epilepsy surgery

Purpose: To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers. Methods: Seven centers participating in an ongoing, prospective multicenter study of resective epilepsy surgery agreed to conform to a detailed protocol regarding presurgical evaluation and surgery. To assess quality assurance, each center independently reviewed 21 randomly selected surgical cases for preoperative study lateralization and localization, and surgical decisions. Interrater reliability was assessed by using intraclass correlation coefficients (ICCs), validated for use with multiple raters, and calculated in a two-way random model based on absolute agreement. Results: Agreement for ICC values: greater than or equal to0.75, excellent; 0.60-0.74, good; 0.40-0.59, fair; less than or equal to0.39, poor. One center was excluded for missing data. Agreement was excellent for extracranial EEG lateralization (0.8039), magnetic resonance imaging (MRI) lateralization (0.9521) and localization (0.9130), Wada lateralization (0.9453), and intracranial EEG localization (0.7905). Agreement was good for extracranial EEG localization (0.7384) and neuropsychological testing lateralization (0.7178) and localization (0.6891). Consensus about the decision to perform intracranial monitoring was fair (0.5397), in part reflecting one center's tendency toward intracranial monitoring. Overall agreements on whether to perform surgery (0.8311) and specific surgery recommended (0.8164) were excellent. Conclusions: High interrater reliability among six epilepsy centers was present for interpretation of most components of presurgical testing. Although consensus for the decision to perform intracranial monitoring was only fair, agreements for the ultimate decision about resective surgery and specific choice of resection were excellent. We believe that this study demonstrates the feasibility of implementing multicenter protocols for neurologic management, especially those involving localization, as well as protocols combining study results with clinical decision making.