Efron, M.; Winget, M.: Query polyrepresentation for ranking retrieval systems without relevance judgments (2010)
0.00
0.0037454618 = product of:
0.014981847 = sum of:
0.014981847 = product of:
0.04494554 = sum of:
0.04494554 = weight(_text_:k in 3469) [ClassicSimilarity], result of:
0.04494554 = score(doc=3469,freq=4.0), product of:
0.13429943 = queryWeight, product of:
3.569778 = idf(docFreq=3384, maxDocs=44218)
0.037621226 = queryNorm
0.33466667 = fieldWeight in 3469, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.569778 = idf(docFreq=3384, maxDocs=44218)
0.046875 = fieldNorm(doc=3469)
0.33333334 = coord(1/3)
0.25 = coord(1/4)
- Abstract
- Ranking information retrieval (IR) systems with respect to their effectiveness is a crucial operation during IR evaluation, as well as during data fusion. This article offers a novel method of approaching the system-ranking problem, based on the widely studied idea of polyrepresentation. The principle of polyrepresentation suggests that a single information need can be represented by many query articulations-what we call query aspects. By skimming the top k (where k is small) documents retrieved by a single system for multiple query aspects, we collect a set of documents that are likely to be relevant to a given test topic. Labeling these skimmed documents as putatively relevant lets us build pseudorelevance judgments without undue human intervention. We report experiments where using these pseudorelevance judgments delivers a rank ordering of IR systems that correlates highly with rankings based on human relevance judgments.
