Domain-Specific Deep Web Sources Discovery

The Web has been rapidly deepened with myriad searchable databases online, where data are hidden behind query interfaces. However, users often have difficulties in finding the right sources and then querying over them in myriad useful databases online. For solving this problem, this paper presents a new method by importing focused crawling technology to automatically accomplish deep web sources discovery. Firstly, locate Web sites for domain-specific data sources based on focused crawling. Secondly, judge whether the web site exists deep web query interface in the former three depths. Lastly, judge whether the deep Web query interface is relevant to a given topic. Importing focused crawling technology makes the identification of deep web query interface locate in a specific domain and capture relative pages to a given topic instead of pursuing high overlay ratios. This method has dramatically reduced the quantity of pages for the crawler to identify deep Web query interfaces.