Forward Decay: A Practical Time Decay Model for Streaming Systems

被引:47
作者
Cormode, Graham [1 ]
Shkapenyuk, Vladislav [1 ]
Srivastava, Divesh [1 ]
Xu, Bojian [2 ]
机构
[1] AT&T Labs Res, Florham Pk, NJ 07932 USA
[2] Iowa State Univ, Dept ECE, Ames, IA USA
来源
ICDE: 2009 IEEE 25TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING, VOLS 1-3 | 2009年
关键词
D O I
10.1109/ICDE.2009.65
中图分类号
TP31 [计算机软件];
学科分类号
081205 [计算机软件];
摘要
Temporal data analysis in data warehouses and data streaming systems often uses time decay to reduce the importance of older tuples, without eliminating their influence, on the results of the analysis. While exponential time decay is commonly used in practice, other decay functions (e.g. polynomial decay) are not, even though they have been identified as useful. We argue that this is because the usual definitions of time decay are "backwards": the decayed weight of a tuple is based on its age, measured backward from the current time. Since this age is constantly changing, such decay is too complex and unwieldy for scalable implementation. In this paper, we propose a new class of "forward" decay functions based on measuring forward from a fixed point in time. We show that this model captures the more practical models already known, such as exponential decay and landmark windows, but also includes a wide class of other types of time decay. We provide efficient algorithms to compute a variety of aggregates and draw samples under forward decay, and show that these are easy to implement scalably. Further, we provide empirical evidence that these can be executed in a production data stream management system with little or no overhead compared to the undecayed computations. Our implementation required no extensions to the query language or the DSMS, demonstrating that forward decay represents a practical model of time decay for systems that deal with time-based data.
引用
收藏
页码:138 / +
页数:2
相关论文
共 36 条
[1]
ABADI D, 2005, SIGMOD
[2]
ALON N, 2005, PODS
[3]
[Anonymous], 2005, Scientific Programming
[4]
[Anonymous], 2006, VLDB
[5]
[Anonymous], 2003, SIGMOD
[6]
[Anonymous], 2004, OSDI
[7]
Arasu A., 2004, PODS
[8]
Babcock B., 2002, SODA
[9]
Busch C., 2007, STACS
[10]
CARNEY D, 2002, VLDB