Scheduling of Recursive and Dynamic Data-Flow Graphs Using Stream Rewriting

Data-flow graphs, consisting of processes (actors) and communication channels, provide an efficient model of computation for analysis and implementation of highly parallel applications. We propose a novel algorithm for scheduling a large number of data-flow actors and also recursively expandable sub-graphs by encoding their state and dependencies as a token stream. The proposed execution model enables global resource sharing, dynamic instantiation of actors and provides lightweight lock-free synchronization. Hence, our approach is most useful for compute-intensive applications with frequently varying and unpredictable data rates. In addition, we present a balanced scheduling algorithm, which restricts the memory usage of dynamic and recursive data-flow graphs, while still maintaining enough concurrency to keep all execution units utilized.