Phobos: a front-end approach to extensible compilers

This paper describes a practical approach for implementing domain-specific languages with extensible compilers. Given a compiler with one or more front-end languages, we introduce the idea of a "generic" front-end that allows the syntactic and semantic specification of domain-specific languages. Phobos, our generic front-end, offers modular language specification, allowing the programmer to define new syntax and semantics incrementally. A key feature of our approach is the use of an open term language that can be used to describe arbitrary syntax, and the use of a term rewriting engine to encode semantic actions. The term language is expressive. Scoping can be defined explicitly, and term rewrites use second-order substitution, allowing the use of higher-order abstract syntax if needed. Given a language specification and a source string, the generic front-end constructs a push-down automaton (PDA) based on the supplied grammar, lexes the source string, and simulates the constructed PDA with the stream of tokens obtained. During parsing, rewrite rules associated, with grammar productions are executed, producing a single term when the PDA accepts. This term is then converted, via further rewriting into a compiler representation and compilation proceeds to generate executable code.