Entropy analysis of natural language written texts

The aim of the present work is to investigate the relative contribution of ordered and stochastic components in natural written texts and examine the influence of text category and language on these. To this end, a binary representation of written texts and the generated symbolic sequences are examined by the standard block entropy analysis and the Shannon and Kolmogorov entropies are obtained. It is found that both entropies are sensitive to both language and text category with the text category sensitivity to follow almost the same trends in both languages (English and Greek) considered. The values of these entropies are compared with those of stochastically generated symbolic sequences and the nature of correlations present in this representation of real written texts is identified.