Linguistic interpretation of human motion based on mental image directed semantic theory

The mental image directed semantic theory (MIDST) has proposed an intermediate knowledge representation scheme based on an omnisensual mental image model. This paper presents a formal language for describing multimedia contents, L_md, whose syntax and semantics are based on MIDST and its application to linguistic interpretation of human motion data obtained through a motion capture system, which we think will serve for linguistic summarization of immense amount of time-sequenced non-linguistic data such as movies.