A weighted view on the partial least-squares algorithm

In this paper it is shown that the Partial Least-Squares (PLS) algorithm for univariate data is equivalent to using a truncated Cayley-Hamilton polynomial expression of degree 1 less than or equal to a less than or equal to r for the matrix inverse ((XX)-X-T)(-1) is an element of R-rxr which is used to compute the least-squares (LS) solution. Furthermore, the a coefficients in this polynomial are computed as the optimal LS solution (minimizing parameters) to the prediction error. The resulting solution is non-iterative. The solution can be expressed in terms of a matrix inverse and is given by B-PLS = K-a((KaXXKa)-X-T-X-T)(-1) (KaXY)-X-T-Y-T where K-a is an element of R-rxa is the controllability (Krylov) matrix for the pair ((XX)-X-T, (XY)-Y-T). The iterative PLS algorithm for computing the orthogonal weighting matrix W-a as presented in the literature, is shown here to be equivalent to computing an orthonormal basis (using, e.g. the QR algorithm) for the column space of K-a. The PLS solution can equivalently be computed as B-PLS = W-a((WaXXWa)-X-T-X-T)(-1) (WaXY)-X-T-Y-T, where W-a is the Q (orthogonal) matrix from the QR decomposition K-a = WaR. Furthermore, we have presented an optimal and non-iterative truncated Cayley-Hamilton polynomial LS solution for multivariate data. The free parameters in this solution is found as the minimizing solution of a prediction error criterion. (C) 2000 Elsevier Science Ltd. All rights reserved.