The Dynamic Web Pages Information Extraction Algorithm Based on Sequence Alignment

In this paper, ´common framework´ is defined as the information which is irrelative to the kernel contents of Web pages and common in Web pages from the same source, such as headers, tails, advertisements, orientations of browsers and flash etc. Sequence alignment is adopted in the information extraction algorithm to detect the common framework. After eliminating the common frameworks from Web pages, the data fields obtained will be more suitable for information extraction. On data-intensive Web pages from real-world Websites, the effects of the alignment parameter on extraction results and the phase of common framework detection on decreasing data quantity and increasing extraction accuracy were tested and evaluated. The experimental results proved the validity of this approach convincingly.