Debole, F.; Sebastiani, F.: ¬An analysis of the relative hardness of Reuters-21578 subsets (2005)
0.03
0.029886894 = product of:
0.08966068 = sum of:
0.08966068 = weight(_text_:systematic in 3456) [ClassicSimilarity], result of:
0.08966068 = score(doc=3456,freq=2.0), product of:
0.28397155 = queryWeight, product of:
5.715473 = idf(docFreq=395, maxDocs=44218)
0.049684696 = queryNorm
0.31573826 = fieldWeight in 3456, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.715473 = idf(docFreq=395, maxDocs=44218)
0.0390625 = fieldNorm(doc=3456)
0.33333334 = coord(1/3)
- Abstract
- The existence, public availability, and widespread acceptance of a standard benchmark for a given information retrieval (IR) task are beneficial to research an this task, because they allow different researchers to experimentally compare their own systems by comparing the results they have obtained an this benchmark. The Reuters-21578 test collection, together with its earlier variants, has been such a standard benchmark for the text categorization (TC) task throughout the last 10 years. However, the benefits that this has brought about have somehow been limited by the fact that different researchers have "carved" different subsets out of this collection and tested their systems an one of these subsets only; systems that have been tested an different Reuters-21578 subsets are thus not readily comparable. In this article, we present a systematic, comparative experimental study of the three subsets of Reuters-21578 that have been most popular among TC researchers. The results we obtain allow us to determine the relative hardness of these subsets, thus establishing an indirect means for comparing TC systems that have, or will be, tested an these different subsets.
