Text data mining applied to clustering with cost effective tools

Classification by reading of corrective action reports at NASA can be a lengthy and labor intensive process. This paper shows that a process requiring several weeks of engineer labor can be reduced to a few hours of analyst labor using commercial and in-house data mining applications. Signal processing theory is used to determine the best cluster based on text to use when searching for common cause problems. A method of determining the high-level clusters is presented, and this is followed by a new technique using Fourier transformations and cross-correlations to determine more refined low-level clusters and new information in the data. Finally, a way to apply these results to situations where the cost of lengthy decisions is different from the rewards for quick, correct decisions is discussed. By developing special in-house software, much of the text data mining can be accomplished without purchasing expensive specialized text mining tools.