A cost-based approach to adaptive resource management in data stream systems

Data stream management systems need to adaptively control their resources, since stream characteristics and query workload may vary over time. In this paper, we investigate an approach to adaptive resource management for continuous sliding-window queries that adjusts window sizes and time granularities to keep resource usage within bounds. These two novel techniques differ from standard load shedding approaches based on sampling, as they ensure exact query answers for given user-defined quality of service specifications, even under query reoptimization. In order to quantify the effects of both techniques on the various operations in a query plan, we develop an appropriate cost model for estimating operator resource allocation in terms of memory usage and processing costs. A thorough experimental study not only validates the accuracy of our cost model but also demonstrates the efficacy and scalability of the proposed techniques.