Ahlgren, P.; Grönqvist, L.: Evaluation of retrieval effectiveness with incomplete relevance data : theoretical and experimental comparison of three measures (2008)
0.01
0.014134051 = product of:
0.084804304 = sum of:
0.084804304 = weight(_text_:ranking in 2032) [ClassicSimilarity], result of:
0.084804304 = score(doc=2032,freq=2.0), product of:
0.20271951 = queryWeight, product of:
5.4090285 = idf(docFreq=537, maxDocs=44218)
0.03747799 = queryNorm
0.4183332 = fieldWeight in 2032, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.4090285 = idf(docFreq=537, maxDocs=44218)
0.0546875 = fieldNorm(doc=2032)
0.16666667 = coord(1/6)
- Abstract
- This paper investigates two relatively new measures of retrieval effectiveness in relation to the problem of incomplete relevance data. The measures, Bpref and RankEff, which do not take into account documents that have not been relevance judged, are compared theoretically and experimentally. The experimental comparisons involve a third measure, the well-known mean uninterpolated average precision. The results indicate that RankEff is the most stable of the three measures when the amount of relevance data is reduced, with respect to system ranking and absolute values. In addition, RankEff has the lowest error-rate.
