Extracting statistical data from free-form text

The authors describe a method for processing free-form text files. The method consists of segregating and separating four physically and logically identifiable regions. The four regions are postprocessed to update three history files that contain information about manufactured products over a period of time. The technique used in processing such files falls under the general category of data segregation and character recognition. It involves the use of logical and mathematical operations in recognizing region boundaries and types of data fields and establishing uniqueness in name recognition. Hashing methods are used, combined with logical matrix multiplication in updating the history files. Sparse formats are used to store multiple large arrays on disks, reducing storage requirements by more than a factor of two. The techniques are implemented using multiprogramming environments in an automated system.