A framework for measuring differences in data characteristics

A data mining algorithm builds a model that captures interesting aspects of the underlying data. We develop a framework for quantifying the difference, called the deviation, between two datasets in terms of the models they induce, In addition to being a quantitative, intuitively interpretable measure of difference, the deviation between two datasets can also be computed very fast, Our framework covers a wide variety of models including frequent itemsets, decision tree classifiers. and clusters, and captures standard measures of deviation such as the misclassification rate and the chi-squared metric as special cases. We also show how statistical techniques can be applied to the deviation measure to assess whether the difference between two models is significant (i.e., whether the underlying datasets have statistically significant differences in their characteristics), and discuss several practical applications. (C) 2002 Elsevier Science (USA).