Efron, M.; Winget, M.: Query polyrepresentation for ranking retrieval systems without relevance judgments (2010)
0.01
0.011090445 = product of:
0.04436178 = sum of:
0.04436178 = product of:
0.08872356 = sum of:
0.08872356 = weight(_text_:aspects in 3469) [ClassicSimilarity], result of:
0.08872356 = score(doc=3469,freq=4.0), product of:
0.20938325 = queryWeight, product of:
4.5198684 = idf(docFreq=1308, maxDocs=44218)
0.046325076 = queryNorm
0.42373765 = fieldWeight in 3469, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
4.5198684 = idf(docFreq=1308, maxDocs=44218)
0.046875 = fieldNorm(doc=3469)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- Ranking information retrieval (IR) systems with respect to their effectiveness is a crucial operation during IR evaluation, as well as during data fusion. This article offers a novel method of approaching the system-ranking problem, based on the widely studied idea of polyrepresentation. The principle of polyrepresentation suggests that a single information need can be represented by many query articulations-what we call query aspects. By skimming the top k (where k is small) documents retrieved by a single system for multiple query aspects, we collect a set of documents that are likely to be relevant to a given test topic. Labeling these skimmed documents as putatively relevant lets us build pseudorelevance judgments without undue human intervention. We report experiments where using these pseudorelevance judgments delivers a rank ordering of IR systems that correlates highly with rankings based on human relevance judgments.
