Signal processing for Big Data

Summary only from given. We live in an era of data deluge. Pervasive sensors collect massive amounts of information on every bit of our lives, churning out enormous streams of raw data in various formats. Mining information from unprecedented volumes of data promises to limit the spread of epidemics and diseases, identify trends in financial markets, learn the dynamics of emergent social-computational systems, and also protect critical infrastructure including the smart grid and the Internet´s backbone network. While Big Data can be definitely perceived as a big blessing, big challenges also arise with large-scale datasets. The sheer volume of data makes it often impossible to run analytics using a central processor and storage, and distributed processing with parallelized multi-processors is preferred while the data themselves are stored in the cloud. As many sources continuously generate data in real time, analytics must often be performed “on-the-fly” and without an opportunity to revisit past entries. Due to their disparate origins, massive datasets are noisy, incomplete, prone to outliers, and vulnerable to cyber-attacks. These effects are amplified if the acquisition and transportation cost per datum is driven to a minimum. Overall, Big Data present challenges in which resources such as time, space, and energy, are intertwined in complex ways with data resources. Given these challenges, ample signal processing opportunities arise. This tutorial lecture outlines ongoing research in novel models applicable to a wide range of Big Data analytics problems, as well as algorithms to handle the practical challenges, while revealing fundamental limits and insights on the mathematical trade-offs involved.