An approach in web content mining for clustering web pages

Nowadays, using web and Internet as a world wide information system faces us with so many data. In this direction, the necessity of accessing some tools for data processing in web level which helps the man intelligently to transform these data into useful knowledge seems so important. Clustering the web pages is one of these techniques. In this paper, a new algorithm has been represented to cluster web pages based on data content. The new algorithm has been suggested based on the expressions and key words existed in web pages, and their bit display a vector and using a new similarity criterion obtained from Cosine and Jaccard similarity criterion. To evaluate the efficacy of suggested algorithm, some pages with five subjects of software engineering, computerized networks, architecture of computer, parallel processing and operating system have been investigated and after preparing a suitable data bed the represented algorithm has been simulated separately through two similarity criteria of Cosine and that of represented in this pager and has been evaluated using Dunn index. The results obtained from simulation show high efficiency of the algorithm proposed in separating web pages and their clustering. The represented algorithm can be used in most of the problems related to clustering web pages.