Web Directory Integration Using Conditional Random Fields

The purpose of integrating web directories is to transfer instances from a source to a target directory. Unlike conventional text categorization, in directory integration, there is extra information about the source directory that can be used to improve the classification accuracy. Many approaches exploit the measured similarity between two corresponding classes to enhance traditional text classifiers. These methods perform well if the topics of two classes are very similar, but they could lead to misclassification if the topics are dissimilar. We propose a directory integration approach based on the conditional random fields (CRFs) model, and model the integration process using a finite-state model. The advantage of using CRFs is that the transition features naturally include information about the relations between classes. Our results show that CRFs outperform conventional text classifiers. In addition, CRFs allow us to apply complex features to integrate the information about the contents of class and their labels. The performance of our approach can be improved by applying these features, especially for instances whose source and target classes are moderately similar