Automatic entity extraction and activity detection for aerospace communications

This paper presents a technical approach to training and execution of an information extraction system for aerospace and other applications. The bulk of this project involved developing algorithms for the automation of training for document classification and entity extraction, using medical data and newswire as testing material. A self-organizing clusters approach to unsupervised document classification was developed as was a ´feed and seed´ approach to entity extraction training. Semi-automatic approaches to training were developed to create an analyst toolkit, which would facilitate porting the system to new domains with some human intervention. Executing the system is accomplished via the metonymic bracketing algorithm and through databases, regular expressions and rules sets. This project´s primary applications goal was to build advanced entity extraction capabilities on top of small-vocabulary speech recognition technology to: 1) improve identification of essential information, such as call signs, headings, tower directives; 2) to provide an automated alerts systems that notifies air traffic of probable errors and corrections in communications; 3) to categorize and route messages based on their content and probable destination; and 4) to automatically complete and clarify partial messages based on long-distance inferencing. These goals are accomplished through a combination of artificial intelligence techniques, beginning with a custom-trained speech recognition system geared towards air traffic under noisy conditions. Using the recognized output, basic information is identified in the communication using easily extended XML-formatted content files, which list information about the essential components of air traffic communication, e.g. plane names, digits, alphabetics. These components are compiled into meaningful entities via general context free grammar (CFG) rulesets that define forms of entities like headings, speeds, call signs, etc. The final stage verifies the completeness of the extracted meaningful entity set and automatically: 1) completes garbled or truncated entities; 2) warns participants of errors and corrections; 3) categorizes the event by activity type, e.g. landing directive, acknowledgement,- etc.; and 4) isolates tower directives. The resulting information is presented to air traffic with the goal of reducing communication errors, facilitating accurate transfer of information and providing a text record of communications. The extracted output can be further processed to provide additional automated alerts or extended to other flight scenarios.