CoBWeb-a crawler for the Brazilian Web

Author

Da Silva, Altigran S. ; Veloso, Eveline A. ; Golgher, Paulo B. ; Ribeiro-Neto, Berthier ; Laender, Alberto H F ; Ziviani, Nivio

Author_Institution

Dept. of Comput. Sci., Univ. Fed. de Minas Gerais, Belo Horizonte, Brazil

fYear

1999

fDate

1999

Firstpage

184

Lastpage

191

Abstract

One of the key components of current Web search engines is the document collector. The paper describes CoBWeb, an automatic document collector whose architecture is distributed and highly scalable. CoBWeb aims at collecting large amounts of documents per time period while observing operational and ethical limits in the crawling process. CoBWeb is part of the SIAM (Information Systems in Mobile Computing Environments) search engine which is being implemented to support the Brazilian Web. Thus, several results related to the Brazilian Web are presented

Keywords

document handling; information resources; information retrieval; mobile computing; search engines; Brazilian Web crawler; CoBWeb; Information Systems in Mobile Computing Environments; SIAM search engine; Web search engines; automatic document collector; crawling process; ethical limits; time period; Argon; Computer science; Crawlers; Electrical capacitance tomography; Internet; Robot kinematics; Scheduling; Search engines; Uniform resource locators; Web search;

fLanguage

English

Publisher

ieee

Conference_Titel

String Processing and Information Retrieval Symposium, 1999 and International Workshop on Groupware

Conference_Location

Cancun

Print_ISBN

0-7695-0268-7

Type

conf

DOI

10.1109/SPIRE.1999.796594

Filename

796594