WAVELENGTH SELECTION IN MULTICOMPONENT NEAR-INFRARED CALIBRATION

Modern scanning (near-)infrared reflectance/absorption (NIR) spectroscopes measure the absorptions or reflectances at a sequence of around 1000 wavelengths. Training data may consist of 10-100 carefully designed sample mixtures for which the true composition of the mixture is either known by formulation or accurately determined by wet chemistry. In future one wishes to predict the true composition from the spectrum. In this paper we compare a simple wavelength selection approach with methods which retain all the wavelengths. It offers a powerful yet simple technique for choosing those wavelengths that are specific to each pure component as against the other components (including the medium) for the varying compositions. In the presence of a defined range of ingredients it thus chooses wavelengths which are highly selective for each particular component. It has the added advantage of selecting wavelengths which are little effected by interaction effects and consequent non-linearities. The calibration data used consist of 125 observations of three sugars, each varying at five levels in a full 5(3) design. The validation set consists of 21 further samples specially selected to have compositions outside the range of the training sample. The selection methods perform much better on this prediction set than methods which retain all the wavelengths, 700 in this case. The leave-one-out cross-validation internal to the calibration data would point to the opposite finding and suggests that such cross-validations may be overly flattering to techniques such as partial least squares and may encourage overfitting. After selection, simple straightforward least squares methods may be used, eschewing the need for 'shrinkage' methods such as partial least squares or ridge regression.