A Practical Approach to Resolving Combination Ambiguity in Chinese Word Segmentation

In Chinese word segmentation task, combination ambiguity is one of challenges not being well settled. The main obstacle exists in the detection of ambiguous words in given texts and their proper segmentations. This paper puts forward a practical approach to automatically collecting ambiguous words and disambiguating based on maximum entropy principle. The experimental result reveals the approach of automatic collection ambiguous words can detect combination ambiguity effectively avoiding arduous manual work. As to the disambiguation based on maximum entropy, we investigate new features grounded on prior and contextual knowledge and achieve promising result