R65-25 Training a Computer to Assign Descriptors to Documents: Experiments in Automatic Indexing

Summary form only given. This work describes a technique for utilizing a computer program to assign to technical papers relevant descriptors from a fixed set of such terms. The authors chose a "representative" sample of about one hundred papers from a collection of 10,000 papers previously indexed by analysts at the Defense Documentation Center. The significant content words (those not on a list of stop words to be ignored) of the title and abstract of each paper were extracted, and paired with all the descriptors for that paper. From all the pairs obtained from this teaching sample, and the relative frequency of occurrence of each descriptor, a co-occurrence value for each pair was computed, and for "validated" descriptors (those appearing at least three times in the teaching sample), this co-occurrence data was retained. The remaining descriptor names were kept on a list of "candidate" descriptors.