Optimistic bias in classification accuracy assessment

There are many sources of both conservative and optimistic bias in classification accuracy assessment. In this Letter, we discuss three sources of optimistic bias: use of training data for accuracy assessment, restriction of reference data sampling to homogeneous areas, and sampling of reference data not independent of training data. The magnitude and direction of bias in classification accuracy estimates depends on the methods used for classification and reference data sampling. However, based on our review of 1994 papers published in three remote sensing journals, we conclude that many studies currently do not report their methods in sufficient detail to enable readers to assess the potential for bias in classification accuracy estimates.