An incremental update strategy in Deep Web

An effective incremental web crawler maintains a local repository of web pages up to date. In this paper, we will introduce an approach to update pages in Deep Web. Unlike traditional studies which mainly concentrate on “important pages” or “refresh”, We classify pages with different ways of calculating the measure of their respective priorities, obtain the coefficient ratio between the derived categories through experimental statistics, and automatically adjust parameters to achieve the incremental update.