Title :
Unknown words analysis in POS tagging of Sinhala language
Author :
Jayaweera, A.J.P.M.P. ; Dias, N.G.J.
Author_Institution :
Virtusa Pvt. Ltd., Colombo, Sri Lanka
Abstract :
Appearance of unknown words is one of the frequently occurring problems facing in part of speech (POS) tagging process, i.e., the words that appear in sentences, but are not contained within the training corpus. New words are continually coined to the language, and people will often use words that are parsing, the system may not expect. This problem get worse when NLP systems are used for more and more on-line computer applications. New words are continually entering the language, Acronyms and proper names are created very often and new nouns and verbs are adding to the language in a surprising rate. So it is impossible to train the tagger for every possible word in the language. So unknown words are non-negligible in POS tagging. Therefore, in order to build a complete tagger, tagger must be incurred with some knowledge of suggesting the tag for an unknown word.
Keywords :
natural language processing; speech processing; NLP system; POS tagging; Sinhala language; natural language processing; part-of-speech tagging; Morphology; Part of Speech tagging; Sinhala Language; Unknown words;
Conference_Titel :
Advances in ICT for Emerging Regions (ICTer), 2014 International Conference on
Print_ISBN :
978-1-4799-7731-4
DOI :
10.1109/ICTER.2014.7083928