DocumentCode
3585849
Title
Unknown words analysis in POS tagging of Sinhala language
Author
Jayaweera, A.J.P.M.P. ; Dias, N.G.J.
Author_Institution
Virtusa Pvt. Ltd., Colombo, Sri Lanka
fYear
2014
Firstpage
270
Lastpage
270
Abstract
Appearance of unknown words is one of the frequently occurring problems facing in part of speech (POS) tagging process, i.e., the words that appear in sentences, but are not contained within the training corpus. New words are continually coined to the language, and people will often use words that are parsing, the system may not expect. This problem get worse when NLP systems are used for more and more on-line computer applications. New words are continually entering the language, Acronyms and proper names are created very often and new nouns and verbs are adding to the language in a surprising rate. So it is impossible to train the tagger for every possible word in the language. So unknown words are non-negligible in POS tagging. Therefore, in order to build a complete tagger, tagger must be incurred with some knowledge of suggesting the tag for an unknown word.
Keywords
natural language processing; speech processing; NLP system; POS tagging; Sinhala language; natural language processing; part-of-speech tagging; Morphology; Part of Speech tagging; Sinhala Language; Unknown words;
fLanguage
English
Publisher
ieee
Conference_Titel
Advances in ICT for Emerging Regions (ICTer), 2014 International Conference on
Print_ISBN
978-1-4799-7731-4
Type
conf
DOI
10.1109/ICTER.2014.7083928
Filename
7083928
Link To Document