Big data: The curse of dimensionality in modeling

The problem of variable selection in system identification of a high dimensional nonlinear non-parametric system is described. The inherent difficulty, the curse of dimensionality, is introduced. Its connections to various topics and research areas are briefly discussed including order determination, pattern recognition, data mining, machine learning, statistical regression and manifold embedding. Finally, two methods, top down and bottom up approaches are described in some details.