A common framework for the unification of neural, chemometric and statistical modeling methods Original Research Article

Extraction of empirical models from measured data is essential for several chemometric and engineering tasks. Selection of an appropriate method for a given task requires deep understanding of the characteristics of a variety of empirical modeling methods that have been derived from diverse fields such as statistics, chemometrics, and artificial intelligence. Unfortunately, the necessary insight into the plethora of empirical modeling methods is not easily available, making the selection process subjective and heuristic, often resulting in inferior empirical models. Furthermore, the properties of various methods are complementary, and combining these methods can result in better models. This paper presents a common framework for understanding the similarities and differences between various empirical modeling methods, and for developing hybrid techniques that combine the best properties of existing methods. The framework is based on representing all empirical modeling methods as a weighted sum of basis functions, and comparing various methods depending on decisions about the nature of the input transformation, type of activation functions, and optimization criteria. All empirical modeling methods transform the inputs by projection on a linear or nonlinear hypersurface or by selecting a subset of variables. The activation functions are of fixed or adaptive shape, and the optimization criteria for determining the model parameters are based on either the input space only, or both input and output space. An overview of several popular methods and an illustrative example are presented to enhance the insight into empirical modeling methods provided by the common framework.