Knowledge discovery from text documents based on paragraph maps

In law, physics, business, and so on, there are lots of documents. The organisation of these documents is essential. The right way to organise the documents reveals quite a lot from the information contents of the document. It is common that text documents are characterised and classified by keywords. The authors usually define these keywords. Nowadays there exists a tremendous amount of uncharacterised text documents, due to the Internet and also to old paper based archives. It is important that the information can be managed and the knowledge can be retrieved. It would be desirable to retrieve the information without reading the document. We propose a new technology based on multilevel hierarchies. Here we concentrate only on the highest level. The technology is based on a hierarchy of self-organizing maps (SOM) and on smart encoding of words. Our experiment with a text document (an annual report) shows that it is possible to separate between different types of paragraphs. It is possible to separate between the original paragraph and the one containing the same words but in random order. It is also possible to categorise the paragraphs or for instance, to find all unauthorised citations of paragraphs within a long text document. The only requirement is that there be a considerable amount of text documents for the training process. Finally the text documents can be classified based on the trained types of paragraphs. This means that unknown documents can be categorised without reading them. This facility can be called knowledge discovery.