A simulation study on comparison of prediction methods when only a few components are relevant

An unconditional expected squared error criterion is used for an overall comparison of 5 different prediction methods: Principal component regression by the size of the eigenvalues (PCR1) and by the size of the t-value (PCR2), partial least squares regression (PLS), restricted principal component regression (RPCR), and modified maximum likelihood regression (MML). Because the distributions of the estimated regression coefficients (looking at the calibration set as random) are unknown or only known asymptotically, a large simulation study is performed. By means of a model based on relevant components and by reducing the number of parameters in the model using the symmetries in the situation, the simulations are designed to cover the major part of the parameter space. The main result is that PCR1, PLS and RPCR are the best prediction methods, the three methods being quite similar, with PLS somewhat better when the irrelevant eigenvalues are large, and PCR1 somewhat better when the irrelevant eigenvalues are small.