Derek Doran, D.; Gokhale, S.S.: ¬A classification framework for web robots (2012)
0.01
0.010875943 = product of:
0.0761316 = sum of:
0.0380658 = weight(_text_:classification in 505) [ClassicSimilarity], result of:
0.0380658 = score(doc=505,freq=4.0), product of:
0.09562149 = queryWeight, product of:
3.1847067 = idf(docFreq=4974, maxDocs=44218)
0.03002521 = queryNorm
0.39808834 = fieldWeight in 505, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.1847067 = idf(docFreq=4974, maxDocs=44218)
0.0625 = fieldNorm(doc=505)
0.0380658 = weight(_text_:classification in 505) [ClassicSimilarity], result of:
0.0380658 = score(doc=505,freq=4.0), product of:
0.09562149 = queryWeight, product of:
3.1847067 = idf(docFreq=4974, maxDocs=44218)
0.03002521 = queryNorm
0.39808834 = fieldWeight in 505, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.1847067 = idf(docFreq=4974, maxDocs=44218)
0.0625 = fieldNorm(doc=505)
0.14285715 = coord(2/14)
- Abstract
- The behavior of modern web robots varies widely when they crawl for different purposes. In this article, we present a framework to classify these web robots from two orthogonal perspectives, namely, their functionality and the types of resources they consume. Applying the classification framework to a year-long access log from the UConn SoE web server, we present trends that point to significant differences in their crawling behavior.