An asynchronous solver for systems of ODEs linked by a directed tree structure

This paper documents our development and evaluation of a numerical solver for systems of sparsely linked ordinary differential equations in which the connectivity between equations is determined by a directed tree. These types of systems arise in distributed hydrological models. The numerical solver is based on dense output Runge–Kutta methods that allow for asynchronous integration. A partition of the system is used to distribute the workload among different processes, enabling a parallel implementation that capitalizes on a distributed memory system. Communication between processes is performed asynchronously. We illustrate the solver capabilities by integrating flow transport equations for a ∼17,000 km2 river basin subdivided into 305,000 sub-watersheds that are interconnected by the river network. Numerical experiments for a few models are performed and the runtimes and scalability on our parallel computer are presented. Efficient numerical integrators such as the one demonstrated here bring closer to reality the goal of implementing fully distributed real-time flood forecasting systems supported by physics based hydrological models and high-quality/high-resolution rainfall products.