Diagram understanding utilizing natural language text

Diagram understanding and its cooperative use with other media are important subjects in both pattern understanding and communication. However, it is quite difficult to understand diagrams without supplementary explanation by other media. For this purpose, the authors propose a new framework for semantic understanding of a diagram by utilizing textual information. In this framework, the elements in a diagram are tightly linked to words or other structure in a natural language text, and the semantic structure of a diagram is interpreted clearly by using natural language information. The obtained result contributes to the construction of hypertext and multimedia