Generating coordinated multimedia explanations

The coordinated multimedia explanation testbed (COMET) is a research system being developed to explore the coordinated generation of multimedia explanations of equipment maintenance and repair procedures. The form and content of all material presented is generated interactively, allowing explanations to be customized for the individual user and situation. COMET´s architecture includes multiple static and dynamic knowledge sources, a content planner, a media coordinator, media generators and a media layout manager. Coordinating multiple media through the development of a single content planner that generates a common content description for both media is emphasized. This content description is annotated by the media coordinator to indicate the assignment of the concepts to be expressed to the media generators. This will ultimately allow cross-references between media and coordinated display layouts that reflect fine-grain relationships among the material presented