A methodology for training and validating a CAD system and potential pitfalls

In this study we first discuss potential pitfalls involved in training a classifier for a CAD system and then propose a methodology for a successful validation of a CAD system. Our approach tries to achieve a balance between performing well on the training data while generalizing well on new cases. We performed several experiments to justify each step of the proposed methodology. As our experimental results suggest, one can safely consider leave-one-patient-out in tuning the classifier and selecting the relevant features as a performance measure. However, the final classifier should always be evaluated on an independent test set. (C) 2004 CARS and Elsevier B.V. All rights reserved.