Extracting Web News Using Tag Path Patterns

How to accurately extract the content of Web news is a popular and significant issue in Web Intelligence. Many Web news sites have similar structures and layout styles, and there are potential correlations between Web content layouts and tag path patterns. Compared with other extraction features, such as HTML tags, literal words and visual features, a tag path pattern not only addresses content segments well, but also has an advantage in the generalization. However, can we accurately extract Web news using only tag path patterns? Motivated by this problem, we propose a PPWIE extraction model. We design an extraction algorithm WEtr using self-defined tag path patterns, and then define a special tag path pattern called the distinguishing tag path pattern. In addition, to tackle the NPC-hard problem in path pattern mining, we propose a polynomial-time (ln|n|+1)-approximation algorithm MPM, in which n indicates the scale of positive samples. Our experiments show that our integration method WEtr+MPM in PPWIE can achieve better performance with more than 98% of precision, recall and the F-score on real world datasets.