Title :
CoBWeb-a crawler for the Brazilian Web
Author :
Da Silva, Altigran S. ; Veloso, Eveline A. ; Golgher, Paulo B. ; Ribeiro-Neto, Berthier ; Laender, Alberto H F ; Ziviani, Nivio
Author_Institution :
Dept. of Comput. Sci., Univ. Fed. de Minas Gerais, Belo Horizonte, Brazil
Abstract :
One of the key components of current Web search engines is the document collector. The paper describes CoBWeb, an automatic document collector whose architecture is distributed and highly scalable. CoBWeb aims at collecting large amounts of documents per time period while observing operational and ethical limits in the crawling process. CoBWeb is part of the SIAM (Information Systems in Mobile Computing Environments) search engine which is being implemented to support the Brazilian Web. Thus, several results related to the Brazilian Web are presented
Keywords :
document handling; information resources; information retrieval; mobile computing; search engines; Brazilian Web crawler; CoBWeb; Information Systems in Mobile Computing Environments; SIAM search engine; Web search engines; automatic document collector; crawling process; ethical limits; time period; Argon; Computer science; Crawlers; Electrical capacitance tomography; Internet; Robot kinematics; Scheduling; Search engines; Uniform resource locators; Web search;
Conference_Titel :
String Processing and Information Retrieval Symposium, 1999 and International Workshop on Groupware
Conference_Location :
Cancun
Print_ISBN :
0-7695-0268-7
DOI :
10.1109/SPIRE.1999.796594
