Entropy of Malayalam language and text compression using Huffman coding

Entropy is a statistical parameter which measures how much information is produced on the average for each letter of a text in the language. Every language normally has certain hidden statistically significant features and certain redundancy. These features can be utilized to form a suitable text compression tool for the optimum use of resources. Being motivated by the language studies of English and other languages based on Shannon theory, an informational analysis of Malayalam language text is done in this paper. Entropy of Malayalam language is calculated and is obtained as 4.8 bits per character. The Malayalam text compressor discussed in this paper, follows Huffman coding technique which takes both Malayalam and English alphabets along with arithmetic numbers and most probable character is represented by less number of bits. It is found that the Huffman compression algorithm achieves a compression ratio of 66 percentage for a standard Malayalam database taken. A comparison is made on compression ratio for different databases taken.