Identity and Variation Spaces: Revisiting the Fisher Linear Discriminant

The Fisher Linear Discriminant (FLD) is commonly used in classification to find a subspace that maximally separates class patterns according to the Fisher Criterion. It was previously proven that a pre-whitening step can be used to truly optimize the Fisher Criterion. In this paper, we study the theoretical properties of the subspaces induced by this whitened FLD. Of the four subspaces induced, two are most important for classification and representation of patterns. We call these Identity Space and Variation Space. We show that only the between-class variation remains in Identity Space, and only the within-class variation remains in Variation Space. Both spaces can be used for decomposition and representation of class data. Moreover, we give sufficient conditions for these spaces to exist. Finally, we also run experiments to show how Identity and Variation Spaces may be used for classification and image synthesis.