Residual Factor Analysis for Text-Independent Speaker Verification

Joint factor analysis (JFA) has become the state-of-the-art technique in the problem of speaker verification. At the same time, the training of eigenvoice matrix seems to be a heavy burden to us, because it requires lots of multi-channel data, which largely determines the performance of the system. In this paper, we first try to exploit an upper bound performance of the JFA system in a non-normal way, and then proposed a new technique which we referred as residual factor analysis (RFA), in which we replace the eigenvoice matrix in JFA system with the residual vector, to remove the heavy burden of training eigenvoice matrix. We tested the proposed technique on the core condition of NIST 2006 speaker recognition evaluation (SRE 06) and obtained equivalent results to JFA system (equal error rate of about 3.99%), while our method requires no extra multi-channel data except some for training eigenchannel matrix.