The performance analysis of a Chi-square similarity measure for topic related clustering of noisy transcripts

The goal of the paper is to present a novel Chi-square similarity measure and assess its performance through comparison with well-known similarity measures such as Cosine, Dice, and Jaccard. The Chi-square similarity measure has been designed to withstand the imperfections of transcribed spoken documents. The major difference of our similarity measure from others consists in the fact that in addition to searching for co-occurring words in documents, we also match informative closeness of common words. We assume that co-occurring words, which had been employed to convey the same information, should have the compatible significance in matching documents. To test it we apply the Chi-square method. Experimental results obtained via using an archive of transcribed news broadcasts demonstrate the high efficacy of the proposed methodology.