Efron, M.; Winget, M.: Query polyrepresentation for ranking retrieval systems without relevance judgments (2010)
0.01
0.014147157 = product of:
0.028294314 = sum of:
0.028294314 = product of:
0.056588627 = sum of:
0.056588627 = weight(_text_:systems in 3469) [ClassicSimilarity], result of:
0.056588627 = score(doc=3469,freq=6.0), product of:
0.16037072 = queryWeight, product of:
3.0731742 = idf(docFreq=5561, maxDocs=44218)
0.052184064 = queryNorm
0.35286134 = fieldWeight in 3469, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
3.0731742 = idf(docFreq=5561, maxDocs=44218)
0.046875 = fieldNorm(doc=3469)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Ranking information retrieval (IR) systems with respect to their effectiveness is a crucial operation during IR evaluation, as well as during data fusion. This article offers a novel method of approaching the system-ranking problem, based on the widely studied idea of polyrepresentation. The principle of polyrepresentation suggests that a single information need can be represented by many query articulations-what we call query aspects. By skimming the top k (where k is small) documents retrieved by a single system for multiple query aspects, we collect a set of documents that are likely to be relevant to a given test topic. Labeling these skimmed documents as putatively relevant lets us build pseudorelevance judgments without undue human intervention. We report experiments where using these pseudorelevance judgments delivers a rank ordering of IR systems that correlates highly with rankings based on human relevance judgments.
