Closing the loop on a complete linkage hierarchical clustering method

To develop a complete linkage hierarchical clustering method that 1) substantially improves upon the accuracy of the standard complete linkage method and 2) can be fully automated or used with minimal operator supervision, the assumptions underlying the standard complete linkage method are unwound, evaluating pairs of data points for linkage is decoupled from constructing cluster sets, and cluster sets are constructed de novo. These design choices make it possible to construct only the cluster sets that correspond to select, possibly non-contiguous levels of an n·(n−1) over 2 + 1-level hierarchical sequence. To construct meaningful cluster sets without constructing an entire hierarchical sequence, a means that uses distance graphs is used to find meaningful levels of such a hierarchical sequence. This paper presents an approach that mathematically captures the graphical relationships that are used to find meaningful levels and integrates the means into the new clustering method. The approach is inexpensive to implement. Consequently, the new clustering method is self-contained and incurs almost no extra cost to determine which cluster sets should be constructed and which should not. Empirical results from four experiments show that the approach does well at finding meaningful levels of hierarchical sequences.