Simultaneous variable selection and outlier detection using a robust genetic algorithm

Given a dataset in which it is known that all spectra are representative, without error, and have matching accurate reference values, there are many tools which exist to determine the best set of variables to use for constructing an inverse model, such as partial least squares (PLS). Likewise, given that the best variables are known a priori, there are many tools that can be used to determine if any samples are outliers, either due to inaccurate reference values, or due to invalid spectra. However, in many real-world situations, the reference values contain error and the spectra are imperfect. In this situation, it is not always possible to determine either the best subset of samples or the best subset of variables. This paper presents a new technique for combining a robust outlier determination method with a genetic algorithm optimized for spectral variable selection. No assumptions are made as to the optimum set of variables or as to the amount and structure of the errors present in either the predictor (X) or predictand (Y) variables. The technique is best suited for datasets which contain redundant information, i.e., datasets from designed experiments with no replicates may not produce optimum results, as the experimental design implicitly assumes there are no outlier data.