Efron, M.; Winget, M.: Query polyrepresentation for ranking retrieval systems without relevance judgments (2010)
0.01
0.010626211 = product of:
0.026565526 = sum of:
0.009978054 = product of:
0.04989027 = sum of:
0.04989027 = weight(_text_:problem in 3469) [ClassicSimilarity], result of:
0.04989027 = score(doc=3469,freq=2.0), product of:
0.17731056 = queryWeight, product of:
4.244485 = idf(docFreq=1723, maxDocs=44218)
0.04177434 = queryNorm
0.28137225 = fieldWeight in 3469, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.244485 = idf(docFreq=1723, maxDocs=44218)
0.046875 = fieldNorm(doc=3469)
0.2 = coord(1/5)
0.016587472 = weight(_text_:of in 3469) [ClassicSimilarity], result of:
0.016587472 = score(doc=3469,freq=12.0), product of:
0.06532493 = queryWeight, product of:
1.5637573 = idf(docFreq=25162, maxDocs=44218)
0.04177434 = queryNorm
0.25392252 = fieldWeight in 3469, product of:
3.4641016 = tf(freq=12.0), with freq of:
12.0 = termFreq=12.0
1.5637573 = idf(docFreq=25162, maxDocs=44218)
0.046875 = fieldNorm(doc=3469)
0.4 = coord(2/5)
- Abstract
- Ranking information retrieval (IR) systems with respect to their effectiveness is a crucial operation during IR evaluation, as well as during data fusion. This article offers a novel method of approaching the system-ranking problem, based on the widely studied idea of polyrepresentation. The principle of polyrepresentation suggests that a single information need can be represented by many query articulations-what we call query aspects. By skimming the top k (where k is small) documents retrieved by a single system for multiple query aspects, we collect a set of documents that are likely to be relevant to a given test topic. Labeling these skimmed documents as putatively relevant lets us build pseudorelevance judgments without undue human intervention. We report experiments where using these pseudorelevance judgments delivers a rank ordering of IR systems that correlates highly with rankings based on human relevance judgments.
- Source
- Journal of the American Society for Information Science and Technology. 61(2010) no.6, S.1081-1091