Kekäläinen, J.; Järvelin, K.: Using graded relevance assessments in IR evaluation (2002)
0.01
0.0052890894 = product of:
0.042312715 = sum of:
0.042312715 = weight(_text_:case in 5225) [ClassicSimilarity], result of:
0.042312715 = score(doc=5225,freq=2.0), product of:
0.1742197 = queryWeight, product of:
4.3964143 = idf(docFreq=1480, maxDocs=44218)
0.03962768 = queryNorm
0.24286987 = fieldWeight in 5225, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.3964143 = idf(docFreq=1480, maxDocs=44218)
0.0390625 = fieldNorm(doc=5225)
0.125 = coord(1/8)
- Abstract
- Kekalainen and Jarvelin use what they term generalized, nonbinary recall and precision measures where recall is the sum of the relevance scores of the retrieved documents divided by the sum of relevance scores of all documents in the data base, and precision is the sum of the relevance scores of the retrieved documents divided by the number of documents where the relevance scores are real numbers between zero and one. Using the In-Query system and a text data base of 53,893 newspaper articles with 30 queries selected from those for which four relevance categories to provide recall measures were available, search results were evaluated by four judges. Searches were done by average key term weight, Boolean expression, and by average term weight where the terms are grouped by a synonym operator, and for each case with and without expansion of the original terms. Use of higher standards of relevance appears to increase the superiority of the best method. Some methods do a better job of getting the highly relevant documents but do not increase retrieval of marginal ones. There is evidence that generalized precision provides more equitable results, while binary precision provides undeserved merit to some methods. Generally graded relevance measures seem to provide additional insight into IR evaluation.
