Exponentially weighted moving average charts for detecting concept drift

被引:263
作者
Ross, Gordon J. [1 ]
Adams, Niall M. [1 ]
Tasoulis, Dimitris K. [1 ]
Hand, David J. [1 ]
机构
[1] Univ London Imperial Coll Sci Technol & Med, Dept Math, London SW7 2AZ, England
关键词
Streaming classification; Concept drift; Change detection;
D O I
10.1016/j.patrec.2011.08.019
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Classifying streaming data requires the development of methods which are computationally efficient and able to cope with changes in the underlying distribution of the stream, a phenomenon known in the literature as concept drift. We propose a new method for detecting concept drift which uses an exponentially weighted moving average (EWMA) chart to monitor the misclassification rate of an streaming classifier. Our approach is modular and can hence be run in parallel with any underlying classifier to provide an additional layer of concept drift detection. Moreover our method is computationally efficient with overhead O(1) and works in a fully online manner with no need to store data points in memory. Unlike many existing approaches to concept drift detection, our method allows the rate of false positive detections to be controlled and kept constant over time. (C) 2011 Published by Elsevier B.V.
引用
收藏
页码:191 / 198
页数:8
相关论文
共 24 条
[21]   Adaptive threshold computation for CUSUM-type procedures in change detection and isolation problems [J].
Verdier, Ghislain ;
Hilgert, Nadine ;
Vila, Jean-Pierre .
COMPUTATIONAL STATISTICS & DATA ANALYSIS, 2008, 52 (09) :4161-4174
[22]   A comparison of state-of-the-art classification techniques for expert automobile insurance claim fraud detection [J].
Viaene, S ;
Derrig, RA ;
Baesens, B ;
Dedene, G .
JOURNAL OF RISK AND INSURANCE, 2002, 69 (03) :373-421
[23]  
Widmer G, 1996, MACH LEARN, V23, P69, DOI 10.1023/A:1018046501280
[24]   EWMA control charts for monitoring high-yield processes based on non-transformed observations [J].
Yeh, Arthur B. ;
Mcgrath, Richard N. ;
Sembower, Mark A. ;
Shen, Qi .
INTERNATIONAL JOURNAL OF PRODUCTION RESEARCH, 2008, 46 (20) :5679-5699