Dimensionality reduction in a connectionist framework

While designing an application system, often we need to minimize the required number of features at least for three reasons: to reduce the cost of the system, to reduce the cost of decision making, and to honor physical constraints imposed by the specific application. Dimensionality reduction can broadly be done in two ways: (i) by replacing the original set of features by a new set of features in a lower dimension (dimensionality reduction through extraction) and (ii) by selecting a subset of the given set of features (dimensionality reduction through selection). In both cases, the reduction process could be supervised or unsupervised. Here using neural networks, first we shall consider dimensionality reduction through extraction and then through selection.