An evaluating method of spider detection techniques by trap

Author

Chunlong, Fan ; Zhouhua, Yu ; Lei, Xu

Author_Institution

Dept. of Comput. Sci., Shenyang Inst. of Aeronaut. Eng., Shenyang, China

Volume

1

fYear

2010

fDate

21-24 May 2010

Abstract

Spider is a program for obtaining internet resources. For monitoring spider visits to your website, Decision Tree, Bayesian Network and other Spider Detection Techniques (SDT) are proposed. At present, the evaluation of these detection techniques mainly relies on manual analysis of web log data to calculate the recall rate and precision rate. In order to avoid subjectivity caused by manual analysis, an Evaluation Method based on Trap detection technique of spider (EMT) is proposed in this paper which can evaluate the detecting capability of SDT. The traps layout information on the website and the process information of users accessing website resources are used to calculate relevant parameters, indicators and error range of EMT according to the binomial distribution theory. The Experiment results indicate that EMT and the artificial analysis method have consistent conclusion.

Keywords

Bayes methods; Web sites; decision trees; online front-ends; search engines; Bayesian network; EMT; Internet resources; SDT; Spider detection technique; Web site; binomial distribution theory; decision tree; trap detection technique; Aerospace engineering; Bayesian methods; Computer science; Decision trees; Humans; Internet; Manuals; Robots; Search engines; Uniform resource locators; accuracy rate; evaluate; recall rate; spider detection; trap;

fLanguage

English

Publisher

ieee

Conference_Titel

Future Computer and Communication (ICFCC), 2010 2nd International Conference on

Conference_Location

Wuhan

Print_ISBN

978-1-4244-5821-9

Type

conf

DOI

10.1109/ICFCC.2010.5497315

Filename

5497315