Automatic Word Segmentation for Chinese Classics of Tea Based on Tree-Pruning

Automatic word-segmentation is vital for the reading, comprehension and translation of classics. However, large amount of special terms, allusions and proper names within the classics make it difficult for word segmentation. Taking classics of tea as the subject of research, a method was proposed using likelihood ratio statistics to decide two-character words candidate, three character words candidates and multi-character words candidates, and then segment classics of tea automatically by tree-pruning algorithm. The computation complexity of the tree-pruning algorithm is O (LN), L is number of the Chinese characters of the longest word. Experiments show it has better results in word-segmentation.