Three validation procedures, single evaluation set, cross-validation and repeated evaluation set, were tested on near-infrared spectrometric data to evaluate the predictive residual standard deviation and the complexity of the regression model based on partial least-squares (PLS) regression. Thirty-six combinations of response variables and predictor variables (originating from three response variables and spectra recorded on the same 60 samples in four laboratories with different instruments) were tested. Each validation method was used with several different percentages of objects in the evaluation sets, from very low percentages (leave-one-out) to 33%. The results show that the frequently used technique of the single evaluation set gives a bad estimate both of the residual standard deviation and of the complexity of PLS model. Cross-validation gives acceptable estimates when at least ten cancellation groups are used. The validation technique based on the repeated evaluation set, with a large number of repetitions of prediction, gives excellent estimates of residual standard deviation and of model complexity, but it requires a very long computing time.
