TSearch: A Self-learning Vertical Search Spider for Travel

A self-learning vertical search spider for travel is presented. This paper focuses on two machine learning methods SNBC (self-learning naive Bayes classifier) and LQNBC (log quotient naive Bayes classifier) for improving search quality and topic relevance. A framework of designing and implementing a vertical spider TSearch with basic general search spider architecture and functions is also showed. TSearch uses SNBC to filter HTML pages and relies on LQNBC to detect unknown travel related Web sites with high precision. The recall and the precision for the classification of texts crawled by TSearch were measured experimentally. These experiments indicate that using LQNBC and SNBC, TSearch can produce promising travel related information for search.