Distributed classification of Persian News (Case study: Hamshahri News dataset)

Classifying the News specifies the most likely topic that the News content refers to it. In this paper, we use distance detection in vector space model for classifying the News articles. In this method, it is calculated distances between weighted frequency vectors of each category, and the News vector determine its category by finding minimum distance with weighted frequency vector of categories. According to volume of the News articles on each topic, extracting keywords, building weighted frequency vectors and determining vector distances are very time consuming operations. So, in order to increase performance, calculation accuracy and decrease execution time, we use MapReduce, a distributed programming model, to implement and execute distributed classification of the News articles. This research is the first attempt to classifying Persian data in distributed manner and results of this research can be used for other text mining areas in any languages. It is worth mentioning that we have successfully implemented our method on the supercomputer of Amirkabir University of Technology.