Small data is the problem

Small data is the limiting issue for signal processing and science in general, especially in the contemporary era of complex systems. Lack of data impacts system identification and the problem is compounded by having insufficient data for system validation. With biological systems, feature spaces for both classification and network representation contain thousands of potential variables. Even when these spaces are dramatically reduced, owing to a paucity of data, the best one can do is identify an uncertainty class of distributions constructed via a combination of prior knowledge and data. We discuss the small sample conundrum and Bayesian approaches to address it.