Research on the application of page segmentation in information retrieval

Existing search engines index web pages as a whole and use them for information retrieval, which leads to irrelevant documents being returned to users. This paper proposes a new indexing approach for solving this problem by 1) using VIPS algorithm for page segmentation, 2) filtering out the function blocks through several heuristic rules, 3) clustering feature blocks into different sub-documents and indexing them respectively. For three user queries, the initial results retrieved from Google are compared with the search results of improved indexing system, which shows that our approach gets a higher performance in terms of precision and F-measure.