Cyclic subspace regression with analysis of the hat matrix

Cyclic subspace regression (CSR) is a new approach to the complex multivariate calibration problem. The simple algorithm produces solutions for principal component regression (PCR), partial least squares (PLS), least squares (LS), and other related intermediate regressions. This paper describes further analysis of CSR and shows that by using hat matrices, CSR regression vectors are formed from a summation of weighted eigenvectors where weights are determined from the hat matrix, singular values, and sample space eigenvectors. Examination of CSR weights for PCR and PLS further documents differences and similarities and provides information to assist in determining prediction rank for PCR and PLS. By redefining CSR in terms of weighted eigenvectors, it can be shown when PLS and PCR produce essentially the same results where minor differences stem from overfitting by PLS. Additionally, weights derived from the hat matrix show when PCR and PLS generate different results and why. Equations are shown for the sample space that reveal PLS to be a method based on oblique projections while PCR uses orthogonal projections. The optimal intermediate CSR model can be identified as well. A near infrared data set is studied and illustrates principles involved.