-
Pal, S.; Mitra, M.; Kamps, J.: Evaluation effort, reliability and reusability in XML retrieval (2011)
0.03
0.026587836 = product of:
0.053175673 = sum of:
0.053175673 = sum of:
0.017662432 = weight(_text_:2 in 4197) [ClassicSimilarity], result of:
0.017662432 = score(doc=4197,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.13642694 = fieldWeight in 4197, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=4197)
0.03551324 = weight(_text_:22 in 4197) [ClassicSimilarity], result of:
0.03551324 = score(doc=4197,freq=2.0), product of:
0.18357785 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05242341 = queryNorm
0.19345059 = fieldWeight in 4197, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4197)
0.5 = coord(1/2)
- Date
- 22. 1.2011 14:20:56
- Source
- Journal of the American Society for Information Science and Technology. 62(2011) no.2, S.375-394
-
Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011)
0.03
0.026587836 = product of:
0.053175673 = sum of:
0.053175673 = sum of:
0.017662432 = weight(_text_:2 in 4540) [ClassicSimilarity], result of:
0.017662432 = score(doc=4540,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.13642694 = fieldWeight in 4540, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=4540)
0.03551324 = weight(_text_:22 in 4540) [ClassicSimilarity], result of:
0.03551324 = score(doc=4540,freq=2.0), product of:
0.18357785 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05242341 = queryNorm
0.19345059 = fieldWeight in 4540, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4540)
0.5 = coord(1/2)
- Date
- 12. 7.2011 18:29:22
- Source
- Journal of documentation. 67(2011) no.2, S.264-278
-
Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019)
0.03
0.026587836 = product of:
0.053175673 = sum of:
0.053175673 = sum of:
0.017662432 = weight(_text_:2 in 5287) [ClassicSimilarity], result of:
0.017662432 = score(doc=5287,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.13642694 = fieldWeight in 5287, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=5287)
0.03551324 = weight(_text_:22 in 5287) [ClassicSimilarity], result of:
0.03551324 = score(doc=5287,freq=2.0), product of:
0.18357785 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05242341 = queryNorm
0.19345059 = fieldWeight in 5287, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=5287)
0.5 = coord(1/2)
- Date
- 20. 1.2015 18:30:22
- Source
- Aslib journal of information management. 71(2019) no.1, S.2-17
-
Reichert, S.; Mayr, P.: Untersuchung von Relevanzeigenschaften in einem kontrollierten Eyetracking-Experiment (2012)
0.01
0.010653973 = product of:
0.021307945 = sum of:
0.021307945 = product of:
0.04261589 = sum of:
0.04261589 = weight(_text_:22 in 328) [ClassicSimilarity], result of:
0.04261589 = score(doc=328,freq=2.0), product of:
0.18357785 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05242341 = queryNorm
0.23214069 = fieldWeight in 328, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=328)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 22. 7.2012 19:25:54
-
Wildemuth, B.; Freund, L.; Toms, E.G.: Untangling search task complexity and difficulty in the context of interactive information retrieval studies (2014)
0.01
0.00887831 = product of:
0.01775662 = sum of:
0.01775662 = product of:
0.03551324 = sum of:
0.03551324 = weight(_text_:22 in 1786) [ClassicSimilarity], result of:
0.03551324 = score(doc=1786,freq=2.0), product of:
0.18357785 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05242341 = queryNorm
0.19345059 = fieldWeight in 1786, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=1786)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 6. 4.2015 19:31:22
-
Ravana, S.D.; Taheri, M.S.; Rajagopal, P.: Document-based approach to improve the accuracy of pairwise comparison in evaluating information retrieval systems (2015)
0.01
0.00887831 = product of:
0.01775662 = sum of:
0.01775662 = product of:
0.03551324 = sum of:
0.03551324 = weight(_text_:22 in 2587) [ClassicSimilarity], result of:
0.03551324 = score(doc=2587,freq=2.0), product of:
0.18357785 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05242341 = queryNorm
0.19345059 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 20. 1.2015 18:30:22
-
Günther, M.: Vermitteln Suchmaschinen vollständige Bilder aktueller Themen? : Untersuchung der Gewichtung inhaltlicher Aspekte von Suchmaschinenergebnissen in Deutschland und den USA (2016)
0.01
0.006244613 = product of:
0.012489226 = sum of:
0.012489226 = product of:
0.024978451 = sum of:
0.024978451 = weight(_text_:2 in 3068) [ClassicSimilarity], result of:
0.024978451 = score(doc=3068,freq=4.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.19293682 = fieldWeight in 3068, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=3068)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Zielsetzung - Vor dem Hintergrund von Suchmaschinenverzerrungen sollte herausgefunden werden, ob sich die von Google und Bing vermittelten Bilder aktueller internationaler Themen in Deutschland und den USA hinsichtlich (1) Vollständigkeit, (2) Abdeckung und (3) Gewichtung der jeweiligen inhaltlichen Aspekte unterscheiden. Forschungsmethoden - Für die empirische Untersuchung wurde eine Methode aus Ansätzen der empirischen Sozialwissenschaften (Inhaltsanalyse) und der Informationswissenschaft (Retrievaltests) entwickelt und angewandt. Ergebnisse - Es zeigte sich, dass Google und Bing in Deutschland und den USA (1) keine vollständigen Bilder aktueller internationaler Themen vermitteln, dass sie (2) auf den ersten Trefferpositionen nicht die drei wichtigsten inhaltlichen Aspekte abdecken, und dass es (3) bei der Gewichtung der inhaltlichen Aspekte keine signifikanten Unterschiede gibt. Allerdings erfahren diese Ergebnisse Einschränkungen durch die Methodik und die Auswertung der empirischen Untersuchung. Schlussfolgerungen - Es scheinen tatsächlich inhaltliche Suchmaschinenverzerrungen vorzuliegen - diese könnten Auswirkungen auf die Meinungsbildung der Suchmaschinennutzer haben. Trotz großem Aufwand bei manueller, und qualitativ schlechteren Ergebnissen bei automatischer Untersuchung sollte dieses Thema weiter erforscht werden.
-
Geist, K.: Qualität und Relevanz von bildungsbezogenen Suchergebnissen bei der Suche im Web (2012)
0.01
0.0052987295 = product of:
0.010597459 = sum of:
0.010597459 = product of:
0.021194918 = sum of:
0.021194918 = weight(_text_:2 in 570) [ClassicSimilarity], result of:
0.021194918 = score(doc=570,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.16371232 = fieldWeight in 570, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.046875 = fieldNorm(doc=570)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Source
- Mitteilungen der Vereinigung Österreichischer Bibliothekarinnen und Bibliothekare. 65(2012) H.2, S.261-276
-
Kutlu, M.; Elsayed, T.; Lease, M.: Intelligent topic selection for low-cost information retrieval evaluation : a new perspective on deep vs. shallow judging (2018)
0.00
0.00499569 = product of:
0.00999138 = sum of:
0.00999138 = product of:
0.01998276 = sum of:
0.01998276 = weight(_text_:2 in 5092) [ClassicSimilarity], result of:
0.01998276 = score(doc=5092,freq=4.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.15434946 = fieldWeight in 5092, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.03125 = fieldNorm(doc=5092)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- While test collections provide the cornerstone for Cranfield-based evaluation of information retrieval (IR) systems, it has become practically infeasible to rely on traditional pooling techniques to construct test collections at the scale of today's massive document collections (e.g., ClueWeb12's 700M+ Webpages). This has motivated a flurry of studies proposing more cost-effective yet reliable IR evaluation methods. In this paper, we propose a new intelligent topic selection method which reduces the number of search topics (and thereby costly human relevance judgments) needed for reliable IR evaluation. To rigorously assess our method, we integrate previously disparate lines of research on intelligent topic selection and deep vs. shallow judging (i.e., whether it is more cost-effective to collect many relevance judgments for a few topics or a few judgments for many topics). While prior work on intelligent topic selection has never been evaluated against shallow judging baselines, prior work on deep vs. shallow judging has largely argued for shallowed judging, but assuming random topic selection. We argue that for evaluating any topic selection method, ultimately one must ask whether it is actually useful to select topics, or should one simply perform shallow judging over many topics? In seeking a rigorous answer to this over-arching question, we conduct a comprehensive investigation over a set of relevant factors never previously studied together: 1) method of topic selection; 2) the effect of topic familiarity on human judging speed; and 3) how different topic generation processes (requiring varying human effort) impact (i) budget utilization and (ii) the resultant quality of judgments. Experiments on NIST TREC Robust 2003 and Robust 2004 test collections show that not only can we reliably evaluate IR systems with fewer topics, but also that: 1) when topics are intelligently selected, deep judging is often more cost-effective than shallow judging in evaluation reliability; and 2) topic familiarity and topic generation costs greatly impact the evaluation cost vs. reliability trade-off. Our findings challenge conventional wisdom in showing that deep judging is often preferable to shallow judging when topics are selected intelligently.
-
Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017)
0.00
0.004415608 = product of:
0.008831216 = sum of:
0.008831216 = product of:
0.017662432 = sum of:
0.017662432 = weight(_text_:2 in 3700) [ClassicSimilarity], result of:
0.017662432 = score(doc=3700,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.13642694 = fieldWeight in 3700, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=3700)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Purpose This paper demonstrates how to apply traditional information retrieval evaluation methods based on standards from the Text REtrieval Conference (TREC) and web search evaluation to all types of modern library information systems including online public access catalogs, discovery systems, and digital libraries that provide web search features to gather information from heterogeneous sources. Design/methodology/approach We apply conventional procedures from information retrieval evaluation to the library information system context considering the specific characteristics of modern library materials. Findings We introduce a framework consisting of five parts: (1) search queries, (2) search results, (3) assessors, (4) testing, and (5) data analysis. We show how to deal with comparability problems resulting from diverse document types, e.g., electronic articles vs. printed monographs and what issues need to be considered for retrieval tests in the library context. Practical implications The framework can be used as a guideline for conducting retrieval effectiveness studies in the library context. Originality/value Although a considerable amount of research has been done on information retrieval evaluation, and standards for conducting retrieval effectiveness studies do exist, to our knowledge this is the first attempt to provide a systematic framework for evaluating the retrieval effectiveness of twenty-first-century library information systems. We demonstrate which issues must be considered and what decisions must be made by researchers prior to a retrieval test.
-
Sarigil, E.; Sengor Altingovde, I.; Blanco, R.; Barla Cambazoglu, B.; Ozcan, R.; Ulusoy, Ö.: Characterizing, predicting, and handling web search queries that match very few or no results (2018)
0.00
0.004415608 = product of:
0.008831216 = sum of:
0.008831216 = product of:
0.017662432 = sum of:
0.017662432 = weight(_text_:2 in 4039) [ClassicSimilarity], result of:
0.017662432 = score(doc=4039,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.13642694 = fieldWeight in 4039, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=4039)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Source
- Journal of the Association for Information Science and Technology. 69(2018) no.2, S.256-270
-
Hider, P.: ¬The search value added by professional indexing to a bibliographic database (2018)
0.00
0.004415608 = product of:
0.008831216 = sum of:
0.008831216 = product of:
0.017662432 = sum of:
0.017662432 = weight(_text_:2 in 4300) [ClassicSimilarity], result of:
0.017662432 = score(doc=4300,freq=2.0), product of:
0.1294644 = queryWeight, product of:
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.05242341 = queryNorm
0.13642694 = fieldWeight in 4300, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.4695914 = idf(docFreq=10170, maxDocs=44218)
0.0390625 = fieldNorm(doc=4300)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 2. 9.2014 19:19:40
