Big Data Processing: Data Flow vs Control Flow (New Benchmarking Methodology)

Big Data processing is becoming a reality in numerous real-world applications. One very important area of research with a rapid growth of data volume is sensor networks. This article discusses the shift in the computing paradigm for Big Data problems and applications. We briefly introduce the Data Flow programming model and then focus on the new benchmarking methodology for Big Data processing. Big Data problems and applications that are suitable for implementation on Data Flow computers should not be measured using the same measures as Control Flow computers. We propose a new benchmarking methodology, which takes into account not only the execution time, but also the power and space, needed to complete the task. Recent research shows that if the Top 500 ranking was based on the new performance measures, Data Flow machines would outperform Control Flow machines. To support the above claims, we present some recent implementations of various algorithms using the Data Flow paradigm, which show considerable speed-ups, power reductions, and space savings over their implementation using the Control Flow paradigm.