Neural networks for latent semantic analysis

Textual data can usually be represented by a large-dimensional matrix. For tasks such as information retrieval or information filtering, it is therefore necessary to reduce the dimensionality. A statistical method called latent semantic analysis (LSA) can be used not only for dimensional reduction but also for transformation of the original space into a much lower-dimensional but much more meaningful feature space, usually called a semantic space. However, LSA is known to consume a lot of both memory and computational time. This paper illustrates how to resolve these problems. Neural networks are proposed that perform LSA in a quasi-parallel fashion, where only one orthogonal factor spanning the semantic space is computed at a time. The memory space required could therefore be significantly reduced. In addition, due to their homogeneous structures and learning rules, implementation of the neural networks could be readily done on a VLSI chip or any parallel processing system. Computer simulation experiments are given that demonstrate the validity and performance of the presented neural networks