Extensibility and other model-based ATR evaluation concepts

This paper introduces concepts that, we hope, will help move the discussion of ATR evaluation in a direction that addresses long standing difficulties associated with getting rest results that are meaningful to the program managers as they compare performance across technologies, to the users as they consider applications, and to the developers as they consider alternative approaches to the many ATR challenges. The paper is motivated by the recent need to independently evaluate an ATR system whose design is model-driven, particularly the DARPA/WL Moving and Stationary Target Acquisition and Recognition (MSTAR) Program. There are two complementary classes of concepts. One class, which we call Performance. includes Accuracy, Extensibility, Robustness, and Utility. These Performance concepts encourage explicit consideration of the relationship between the test data, the training data, and data from modeled conditions. The other class, which we call Cost includes Efficiency, Scalability, and Synthetic Trainability. Cost concepts help bring out some of the unique characteristics of the costs associated with ATR design and operation.