Title :
Cooperation among Web Crawlers by Using Contract Net Protocol
Author :
Zhang, Zhen ; Du, Yajun ; Li, Chengjie ; Li, Haiming
Author_Institution :
Dept. of Comput. & Math., Xihua Univ., Chengdu, China
Abstract :
There are many previous researches on focused crawlers. Most of the crawlers are bionic: they gain energy for having scratched relevant pates to the query and lose energy otherwise; they reproduce when energy is high and they die when energy is out. This causes some crawlers in irrelevant area to die so relevant pages that are far to Seed Url may not be scratched and the crawlers´ recall rate may be influenced. In this paper we adopt Contract Net Protocol to solve this problem in a distributed environment.
Keywords :
Internet; protocols; query processing; Web crawlers; contract net protocol; distributed environment; query processing; Conference management; Contracts; Crawlers; IEEE news; Information management; Information technology; Internet; Protocols; Uniform resource locators; Web pages;
Conference_Titel :
Future Information Technology and Management Engineering, 2009. FITME '09. Second International Conference on
Conference_Location :
Sanya
Print_ISBN :
978-1-4244-5339-9
DOI :
10.1109/FITME.2009.27