Derek Doran, D.; Gokhale, S.S.: ¬A classification framework for web robots (2012)
0.00
0.0016410447 = product of:
0.009025746 = sum of:
0.0062481174 = weight(_text_:a in 505) [ClassicSimilarity], result of:
0.0062481174 = score(doc=505,freq=8.0), product of:
0.030653298 = queryWeight, product of:
1.153047 = idf(docFreq=37942, maxDocs=44218)
0.026584605 = queryNorm
0.20383182 = fieldWeight in 505, product of:
2.828427 = tf(freq=8.0), with freq of:
8.0 = termFreq=8.0
1.153047 = idf(docFreq=37942, maxDocs=44218)
0.0625 = fieldNorm(doc=505)
0.0027776284 = weight(_text_:s in 505) [ClassicSimilarity], result of:
0.0027776284 = score(doc=505,freq=2.0), product of:
0.028903782 = queryWeight, product of:
1.0872376 = idf(docFreq=40523, maxDocs=44218)
0.026584605 = queryNorm
0.09609913 = fieldWeight in 505, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
1.0872376 = idf(docFreq=40523, maxDocs=44218)
0.0625 = fieldNorm(doc=505)
0.18181819 = coord(2/11)
- Abstract
- The behavior of modern web robots varies widely when they crawl for different purposes. In this article, we present a framework to classify these web robots from two orthogonal perspectives, namely, their functionality and the types of resources they consume. Applying the classification framework to a year-long access log from the UConn SoE web server, we present trends that point to significant differences in their crawling behavior.
- Source
- Journal of the American Society for Information Science and Technology. 63(2012) no.12, S.2549-2554,
- Type
- a