-
Ravana, S.D.; Taheri, M.S.; Rajagopal, P.: Document-based approach to improve the accuracy of pairwise comparison in evaluating information retrieval systems (2015)
0.03
0.03482235 = product of:
0.0696447 = sum of:
0.0696447 = sum of:
0.03890366 = weight(_text_:t in 2587) [ClassicSimilarity], result of:
0.03890366 = score(doc=2587,freq=2.0), product of:
0.17876579 = queryWeight, product of:
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.04537884 = queryNorm
0.21762364 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.030741034 = weight(_text_:22 in 2587) [ClassicSimilarity], result of:
0.030741034 = score(doc=2587,freq=2.0), product of:
0.15890898 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04537884 = queryNorm
0.19345059 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.5 = coord(1/2)
- Abstract
- Purpose The purpose of this paper is to propose a method to have more accurate results in comparing performance of the paired information retrieval (IR) systems with reference to the current method, which is based on the mean effectiveness scores of the systems across a set of identified topics/queries. Design/methodology/approach Based on the proposed approach, instead of the classic method of using a set of topic scores, the documents level scores are considered as the evaluation unit. These document scores are the defined document's weight, which play the role of the mean average precision (MAP) score of the systems as a significance test's statics. The experiments were conducted using the TREC 9 Web track collection. Findings The p-values generated through the two types of significance tests, namely the Student's t-test and Mann-Whitney show that by using the document level scores as an evaluation unit, the difference between IR systems is more significant compared with utilizing topic scores. Originality/value Utilizing a suitable test collection is a primary prerequisite for IR systems comparative evaluation. However, in addition to reusable test collections, having an accurate statistical testing is a necessity for these evaluations. The findings of this study will assist IR researchers to evaluate their retrieval systems and algorithms more accurately.
- Date
- 20. 1.2015 18:30:22
-
Becks, D.; Mandl, T.; Womser-Hacker, C.: Spezielle Anforderungen bei der Evaluierung von Patent-Retrieval-Systemen (2010)
0.02
0.01925633 = product of:
0.03851266 = sum of:
0.03851266 = product of:
0.07702532 = sum of:
0.07702532 = weight(_text_:t in 4667) [ClassicSimilarity], result of:
0.07702532 = score(doc=4667,freq=4.0), product of:
0.17876579 = queryWeight, product of:
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.04537884 = queryNorm
0.4308728 = fieldWeight in 4667, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.0546875 = fieldNorm(doc=4667)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Source
- Information und Wissen: global, sozial und frei? Proceedings des 12. Internationalen Symposiums für Informationswissenschaft (ISI 2011) ; Hildesheim, 9. - 11. März 2011. Hrsg.: J. Griesbaum, T. Mandl u. C. Womser-Hacker
-
Mandl, T.: Evaluierung im Information Retrieval : die Hildesheimer Antwort auf aktuelle Herausforderungen der globalisierten Informationsgesellschaft (2010)
0.02
0.015561464 = product of:
0.031122928 = sum of:
0.031122928 = product of:
0.062245857 = sum of:
0.062245857 = weight(_text_:t in 4011) [ClassicSimilarity], result of:
0.062245857 = score(doc=4011,freq=2.0), product of:
0.17876579 = queryWeight, product of:
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.04537884 = queryNorm
0.34819782 = fieldWeight in 4011, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.0625 = fieldNorm(doc=4011)
0.5 = coord(1/2)
0.5 = coord(1/2)
-
Tamine, L.; Chouquet, C.; Palmer, T.: Analysis of biomedical and health queries : lessons learned from TREC and CLEF evaluation benchmarks (2015)
0.01
0.009725915 = product of:
0.01945183 = sum of:
0.01945183 = product of:
0.03890366 = sum of:
0.03890366 = weight(_text_:t in 2341) [ClassicSimilarity], result of:
0.03890366 = score(doc=2341,freq=2.0), product of:
0.17876579 = queryWeight, product of:
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.04537884 = queryNorm
0.21762364 = fieldWeight in 2341, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.0390625 = fieldNorm(doc=2341)
0.5 = coord(1/2)
0.5 = coord(1/2)
-
Reichert, S.; Mayr, P.: Untersuchung von Relevanzeigenschaften in einem kontrollierten Eyetracking-Experiment (2012)
0.01
0.00922231 = product of:
0.01844462 = sum of:
0.01844462 = product of:
0.03688924 = sum of:
0.03688924 = weight(_text_:22 in 328) [ClassicSimilarity], result of:
0.03688924 = score(doc=328,freq=2.0), product of:
0.15890898 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04537884 = queryNorm
0.23214069 = fieldWeight in 328, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=328)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 22. 7.2012 19:25:54
-
Kutlu, M.; Elsayed, T.; Lease, M.: Intelligent topic selection for low-cost information retrieval evaluation : a new perspective on deep vs. shallow judging (2018)
0.01
0.007780732 = product of:
0.015561464 = sum of:
0.015561464 = product of:
0.031122928 = sum of:
0.031122928 = weight(_text_:t in 5092) [ClassicSimilarity], result of:
0.031122928 = score(doc=5092,freq=2.0), product of:
0.17876579 = queryWeight, product of:
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.04537884 = queryNorm
0.17409891 = fieldWeight in 5092, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.9394085 = idf(docFreq=2338, maxDocs=44218)
0.03125 = fieldNorm(doc=5092)
0.5 = coord(1/2)
0.5 = coord(1/2)
-
Pal, S.; Mitra, M.; Kamps, J.: Evaluation effort, reliability and reusability in XML retrieval (2011)
0.01
0.0076852585 = product of:
0.015370517 = sum of:
0.015370517 = product of:
0.030741034 = sum of:
0.030741034 = weight(_text_:22 in 4197) [ClassicSimilarity], result of:
0.030741034 = score(doc=4197,freq=2.0), product of:
0.15890898 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04537884 = queryNorm
0.19345059 = fieldWeight in 4197, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4197)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 22. 1.2011 14:20:56
-
Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011)
0.01
0.0076852585 = product of:
0.015370517 = sum of:
0.015370517 = product of:
0.030741034 = sum of:
0.030741034 = weight(_text_:22 in 4540) [ClassicSimilarity], result of:
0.030741034 = score(doc=4540,freq=2.0), product of:
0.15890898 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04537884 = queryNorm
0.19345059 = fieldWeight in 4540, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4540)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 12. 7.2011 18:29:22
-
Wildemuth, B.; Freund, L.; Toms, E.G.: Untangling search task complexity and difficulty in the context of interactive information retrieval studies (2014)
0.01
0.0076852585 = product of:
0.015370517 = sum of:
0.015370517 = product of:
0.030741034 = sum of:
0.030741034 = weight(_text_:22 in 1786) [ClassicSimilarity], result of:
0.030741034 = score(doc=1786,freq=2.0), product of:
0.15890898 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04537884 = queryNorm
0.19345059 = fieldWeight in 1786, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=1786)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 6. 4.2015 19:31:22
-
Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019)
0.01
0.0076852585 = product of:
0.015370517 = sum of:
0.015370517 = product of:
0.030741034 = sum of:
0.030741034 = weight(_text_:22 in 5287) [ClassicSimilarity], result of:
0.030741034 = score(doc=5287,freq=2.0), product of:
0.15890898 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04537884 = queryNorm
0.19345059 = fieldWeight in 5287, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=5287)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 20. 1.2015 18:30:22
