Noise removing from Web pages using neural network

With the exponentially growing amount of information available on the Internet, an effective technique for users to discern the useful information from the unnecessary information is urgently required. Cleaning web pages for web data extraction becomes critical for improving performance of information retrieval and information extraction. So, we investigate to remove various noise patterns in Web pages instead of extracting relevant content from Web pages to get main content information. In this paper, we propose an approach that detect multiple noise patterns and remove these noise patterns from Web pages of any Web sites. The method first build DOM tree for any Web page. Our approach is based on the basic idea of Case-Based Reasoning (CBR) to find noise pattern in current Web page by matching similar noise pattern kept in Case-Based. We also apply a back propagation neural network algorithm to classify the stored various noise patterns by matching similar noise data in current Web page. We have implemented our method on several commercial Web sites and News Web sites to evaluate the performance and improvement of our approach. Experiments show that results leads to a more accurate and effectiveness of the approach.