DocumentCode
2227648
Title
A data-mining approach for optimizing performance of an incremental crawler
Author
Bullot, Hadrien ; Gupta, S.K. ; Mohania, M.K.
Author_Institution
Sch. of Comput. & Commun. Sci., Swiss Fed. Inst. of Technol., Lausanne, Switzerland
fYear
2003
fDate
13-17 Oct. 2003
Firstpage
610
Lastpage
615
Abstract
Crawlers visit the Web to maintain a local repository of Web pages up to date. We introduce another perspective to build an effective incremental crawler. Based on previous work in this field, we study how we can improve the performance of a crawler using data-mining. The information collected from the users can help the crawler to know which are the popular pages and to revisit them as soon as possible.
Keywords
Internet; data mining; optimisation; search engines; Web page; data-mining; incremental crawler performance optimization; search engine; Bandwidth; Computer science; Crawlers; Data analysis; Data mining; Databases; Search engines; Uniform resource locators; Web pages; World Wide Web;
fLanguage
English
Publisher
ieee
Conference_Titel
Web Intelligence, 2003. WI 2003. Proceedings. IEEE/WIC International Conference on
Print_ISBN
0-7695-1932-6
Type
conf
DOI
10.1109/WI.2003.1241279
Filename
1241279
Link To Document