Speaker clustering using vector quantization and spectral clustering

We present a speaker clustering method for conversational speech recordings that contain short utterances from multiple speakers. The proposed method represents a speech segment with a vector of VQ code frequencies and uses a cosine between two vectors as their similarity measure. The clustering is performed by a spectral clustering algorithm with cluster number estimation based on an eigen structure of the similarity matrix. We conducted experiments on five test sets with different utterance length distributions to compare the proposed method with the conventional approach based on a hierarchical agglomerative clustering using BIC stopping criterion. The results show that the proposed method significantly outperforms the conventional one in speaker diarization error rate and purity metrics.