Improving sentence recognition in stochastic context-free grammars

This paper introduces an improved stochastic context-free language recognizer. The algorithm is basically a best-first search on the state space of dotted rules, following Earley´s (1986) notation. Using heuristic merit functions; the algorithm produces the possible successors of a node (representing a state) and proceeds by exploring the most promising generated node. Two simple grammar independent heuristics are introduced: (1) ensuring convergence to a solution; and (2) guaranteeing the choice of the solution with the highest probability. It is shown that the algorithm outperforms Earley´s method in both time and space domains. Several test grammars are used to illustrate the method and show its superior performance