Analyzing the information entropy of states to optimize the number of states in an HMM-based off-line handwritten Arabic word recognizer

HMM is one of the most popular methods to model sequential signals and plays a significant role in the field of off-line handwritten Arabic word recognition research. However, the structure of an HMM including the number of states has to be determined initially and can hardly be updated during the training process. A novel analytic algorithm based on the information entropy of states in an HMM to optimize the number of states will be proposed in this paper. Information entropy is defined as an evaluation criterion of the activity of a state. According to principle of maximum entropy, states with minor information entropy do not possess so enough capability to represent actual observations that they should be deleted. Experiments on IFN/ENIT database show that the algorithm in this paper can bring approximately 3%-6% increase to correct recognition rate from the best performance of system with constant states.