On semantic evaluation of text clustering algorithms

In this paper, we investigate the problem of quality analysis of clustering results using semantic annotations given by experts. In previous work we proposed a novel approach to construction of evaluation measure, called SEE (Semantic Evaluation by Exploration), which is an extension of the existing methods such as Rand Index or Normalized Mutual Information. In this paper we present some further extensions as well as some theoretical properties of the of the proposed measure. We illustrate the proposed evaluation method on documents in INFONA document retrieval system. We compare different search result clustering algorithms using the proposed measure.