Programming language support for writing fault-tolerant distributed software

Good programming language support can simplify the task of writing fault-tolerant distributed software. Here, an approach to providing such support is described in which a general high-level distributed programming language is augmented with mechanisms for fault tolerance. Unlike approaches based on sequential languages or specialized languages oriented towards a given fault-tolerance technique, this approach gives the programmer a high level of abstraction, while still maintaining flexibility and execution efficiency. The paper first describes a programming model that captures the important characteristics that should be supported by a programming language of this type. It then presents a realization of this approach in the form of FT-SR, a programming language that augments the SR distributed programming language with features for replication, recovery, and failure notification. In addition to outlining these extensions, an example program consisting of a data manager and its associated stable storage is given. Finally, an implementation of the language that uses the x-kernel and runs standalone on a network of Sun workstations is discussed. The overall structure and several of the algorithms used in the runtime are interesting in their own right