Characterization of empirical discrepancy evaluation measures

The "quality curve" concept is proposed to characterize the performance of an empirical discrepancy evaluation measure when it is used to compare color edge detection algorithms. This "quality curve" concept is independent of any automatic thresholding algorithm. A simple visual analysis of the "quality curve" allows possible drawbacks of the evaluation measure to be detected. Five classical evaluation measures and ten color edge detection algorithms have been used to confirm the usefulness of the quality curve analysis. Most evaluation measures show drawbacks when they are applied to several color edge detectors. In this case, these measures should not be used to compare that set of color edge detectors. Nevertheless, a less-cited evaluation measure gives the best performance when it is applied to color edge detectors. (C) 2003 Elsevier B.V. All rights reserved.