This article analyzes the evaluation of approximation accuracy in on-line applications. In particular, it is first shown that the most commonly used approximation accuracy evaluation method (e,g., analysis of training or tracking error) is not in itself sufficient to demonstrate proper function approximation, In spite of this, many articles use tracking (training) errors as the means to demonstrate successful function approximation. This article presents two alternative methods for the evaluation of on-line performance. Related issues are probably approximately correct learning from statistics and persistence of excitation from adaptive control.