Comparison of optimization algorithms for piecewise linear discriminant analysis: application to Fourier transform infrared remote sensing measurements

Simplex optimization, simulated annealing, generalized simulated annealing, genetic algorithms, and a Simplex-genetic algorithm hybrid are compared for their ability to optimize piecewise linear discriminants. Nonparametric piecewise linear discriminant analysis (PLDA) is employed here to develop an automated detection scheme for Fourier transform infrared remote sensing interferogram data. Piecewise linear discriminants are computed and optimized for interferograms collected when sulfur hexafluoride, acetone, and methanol were present in the optical path of the spectrometer. The success of this methodology is based on the proper positioning of the discriminant. Discriminant optimization is a challenging problem due to the tendency for local optima to occur on the response surface and the large number of variables being optimized. Among the methods compared in the context of the remote sensing application, Simplex optimization is found to be the best method for discriminant optimization. Discriminants positioned using Simplex optimization are observed to consistently outperform discriminants positioned using algorithms based on simulated annealing and genetic algorithms. The justification for this result is based on three arguments. First, a good starting point for the optimization is available in the PLDA application and only a limited number of response function evaluations are possible due to the requirement of the remote sensing application. These features create an optimization problem that favors Simplex optimization. Second, the response surface is smooth near the starting point and becomes increasingly rugged as the optimal region is approached. Simplex could move quickly at first and then move from local optima through the use of a kick operator. Third, the variables being optimized are highly correlated with each other. The effect of a variable on the response function score is highly dependent on the current values of the other variables. Epistatic optimization problems such as this are very difficult for genetic algorithms but do not impact the performance of Simplex optimization.