Empirical modeling of very large data sets using neural networks

Building empirical predictive models from very large data sets is challenging. One has to deal both with the `curse of dimensionality´ (hundreds or thousands of variables) and with `too many records´ (many thousands of instances). While neural networks [Rumelhart, et al., 1986] are widely recognized as universal function approximators [Cybenko, 1989], their training time rapidly increases with the number of variables and instances. I discuss practical methods for overcoming this problem so that neural network models can be developed for very large databases. The methods include: Dimensionality reduction with neural net modeling, PLS modeling, and bottleneck neural networks; Sub-sampling and re-sampling with many smaller data sets to reduce training time; Committee of networks to make the final prediction more robust and to estimate its uncertainty