An Architectural Framework of a Personalized Web Crawler based on User Interests

TheWorldWideWeb (WWW) is overwhelmed with information which can not be assimilatedby the normal users without the use of search tools. The traditional search returns thousands of resultsfor a single search query making the search and surfing experience cumbersome. This drawback hastriggered the need for implementing personalized search tools. In this paper, a novel architecture is proposedto gather pages that are relevant to a particular user or group of users. The system consists of threemodules: input, crawling and feedback. The input module is integrated with topic suggestion componentextracting search query terms and representative documents from different sources. The crawling moduleis realized with intelligent multi-agent system for prioritizing the download of appropriate URLs. Therelevance of the documents is computed based on interests of the users. While rendering the results, theuser gives feedback and the system is compared to different crawler implementations. The empirical resultsclearly suggest the advantage of using topic suggestion component and computation of personalizedrelevance score in terms of harvest ratio and coverage