Efron, M.; Winget, M.: Query polyrepresentation for ranking retrieval systems without relevance judgments (2010)
0.00
0.0015337638 = product of:
0.012270111 = sum of:
0.012270111 = product of:
0.03681033 = sum of:
0.03681033 = weight(_text_:problem in 3469) [ClassicSimilarity], result of:
0.03681033 = score(doc=3469,freq=2.0), product of:
0.13082431 = queryWeight, product of:
4.244485 = idf(docFreq=1723, maxDocs=44218)
0.030822188 = queryNorm
0.28137225 = fieldWeight in 3469, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.244485 = idf(docFreq=1723, maxDocs=44218)
0.046875 = fieldNorm(doc=3469)
0.33333334 = coord(1/3)
0.125 = coord(1/8)
- Abstract
- Ranking information retrieval (IR) systems with respect to their effectiveness is a crucial operation during IR evaluation, as well as during data fusion. This article offers a novel method of approaching the system-ranking problem, based on the widely studied idea of polyrepresentation. The principle of polyrepresentation suggests that a single information need can be represented by many query articulations-what we call query aspects. By skimming the top k (where k is small) documents retrieved by a single system for multiple query aspects, we collect a set of documents that are likely to be relevant to a given test topic. Labeling these skimmed documents as putatively relevant lets us build pseudorelevance judgments without undue human intervention. We report experiments where using these pseudorelevance judgments delivers a rank ordering of IR systems that correlates highly with rankings based on human relevance judgments.