Semantic HMC: A Predictive Model Using Multi-label Classification for Big Data

One of the biggest challenges in Big Data is the exploitation of Value from large volume of data. To exploit value one must focus on extracting knowledge from Big Data sources. In this paper we present a new simple but highly scalable process to automatically learn the label hierarchy from huge sets of unstructured text. We aim to extract knowledge from these sources using a Hierarchical Multi-Label Classification process called Semantic HMC. Five steps compose the Semantic HMC: Indexation, Vectorization, Hierarchization, Resolution and Realization. The first three steps construct the label hierarchy from data sources. The last two steps classify new items according to the hierarchy labels. To perform the classification without heavily relying on the user, the process is unsupervised, where no thesaurus or label examples are required. The process is implemented in a scalable and distributed platform to process Big Data.