-
Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017)
0.01
0.014126732 = product of:
0.042380195 = sum of:
0.042380195 = weight(_text_:electronic in 3700) [ClassicSimilarity], result of:
0.042380195 = score(doc=3700,freq=2.0), product of:
0.19623034 = queryWeight, product of:
3.9095051 = idf(docFreq=2409, maxDocs=44218)
0.05019314 = queryNorm
0.21597168 = fieldWeight in 3700, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.9095051 = idf(docFreq=2409, maxDocs=44218)
0.0390625 = fieldNorm(doc=3700)
0.33333334 = coord(1/3)
- Abstract
- Purpose This paper demonstrates how to apply traditional information retrieval evaluation methods based on standards from the Text REtrieval Conference (TREC) and web search evaluation to all types of modern library information systems including online public access catalogs, discovery systems, and digital libraries that provide web search features to gather information from heterogeneous sources. Design/methodology/approach We apply conventional procedures from information retrieval evaluation to the library information system context considering the specific characteristics of modern library materials. Findings We introduce a framework consisting of five parts: (1) search queries, (2) search results, (3) assessors, (4) testing, and (5) data analysis. We show how to deal with comparability problems resulting from diverse document types, e.g., electronic articles vs. printed monographs and what issues need to be considered for retrieval tests in the library context. Practical implications The framework can be used as a guideline for conducting retrieval effectiveness studies in the library context. Originality/value Although a considerable amount of research has been done on information retrieval evaluation, and standards for conducting retrieval effectiveness studies do exist, to our knowledge this is the first attempt to provide a systematic framework for evaluating the retrieval effectiveness of twenty-first-century library information systems. We demonstrate which issues must be considered and what decisions must be made by researchers prior to a retrieval test.
-
Munkelt, J.; Schaer, P.; Lepsky, K.: Towards an IR test collection for the German National Library (2018)
0.01
0.0132371 = product of:
0.0397113 = sum of:
0.0397113 = product of:
0.0794226 = sum of:
0.0794226 = weight(_text_:publishing in 4311) [ClassicSimilarity], result of:
0.0794226 = score(doc=4311,freq=2.0), product of:
0.24522576 = queryWeight, product of:
4.885643 = idf(docFreq=907, maxDocs=44218)
0.05019314 = queryNorm
0.32387543 = fieldWeight in 4311, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.885643 = idf(docFreq=907, maxDocs=44218)
0.046875 = fieldNorm(doc=4311)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Abstract
- Automatic content indexing is one of the innovations that are increasingly changing the way libraries work. In theory, it promises a cataloguing service that would hardly be possible with humans in terms of speed, quantity and maybe quality. The German National Library (DNB) has also recognised this potential and is increasingly relying on the automatic indexing of their catalogue content. The DNB took a major step in this direction in 2017, which was announced in two papers. The announcement was rather restrained, but the content of the papers is all the more explosive for the library community: Since September 2017, the DNB has discontinued the intellectual indexing of series Band H and has switched to an automatic process for these series. The subject indexing of online publications (series O) has been purely automatical since 2010; from September 2017, monographs and periodicals published outside the publishing industry and university publications will no longer be indexed by people. This raises the question: What is the quality of the automatic indexing compared to the manual work or in other words to which degree can the automatic indexing replace people without a signi cant drop in regards to quality?
-
Reichert, S.; Mayr, P.: Untersuchung von Relevanzeigenschaften in einem kontrollierten Eyetracking-Experiment (2012)
0.01
0.0068004774 = product of:
0.020401431 = sum of:
0.020401431 = product of:
0.040802862 = sum of:
0.040802862 = weight(_text_:22 in 328) [ClassicSimilarity], result of:
0.040802862 = score(doc=328,freq=2.0), product of:
0.17576782 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05019314 = queryNorm
0.23214069 = fieldWeight in 328, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=328)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Date
- 22. 7.2012 19:25:54
-
Pal, S.; Mitra, M.; Kamps, J.: Evaluation effort, reliability and reusability in XML retrieval (2011)
0.01
0.0056670653 = product of:
0.017001195 = sum of:
0.017001195 = product of:
0.03400239 = sum of:
0.03400239 = weight(_text_:22 in 4197) [ClassicSimilarity], result of:
0.03400239 = score(doc=4197,freq=2.0), product of:
0.17576782 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05019314 = queryNorm
0.19345059 = fieldWeight in 4197, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4197)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Date
- 22. 1.2011 14:20:56
-
Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011)
0.01
0.0056670653 = product of:
0.017001195 = sum of:
0.017001195 = product of:
0.03400239 = sum of:
0.03400239 = weight(_text_:22 in 4540) [ClassicSimilarity], result of:
0.03400239 = score(doc=4540,freq=2.0), product of:
0.17576782 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05019314 = queryNorm
0.19345059 = fieldWeight in 4540, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4540)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Date
- 12. 7.2011 18:29:22
-
Wildemuth, B.; Freund, L.; Toms, E.G.: Untangling search task complexity and difficulty in the context of interactive information retrieval studies (2014)
0.01
0.0056670653 = product of:
0.017001195 = sum of:
0.017001195 = product of:
0.03400239 = sum of:
0.03400239 = weight(_text_:22 in 1786) [ClassicSimilarity], result of:
0.03400239 = score(doc=1786,freq=2.0), product of:
0.17576782 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05019314 = queryNorm
0.19345059 = fieldWeight in 1786, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=1786)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Date
- 6. 4.2015 19:31:22
-
Ravana, S.D.; Taheri, M.S.; Rajagopal, P.: Document-based approach to improve the accuracy of pairwise comparison in evaluating information retrieval systems (2015)
0.01
0.0056670653 = product of:
0.017001195 = sum of:
0.017001195 = product of:
0.03400239 = sum of:
0.03400239 = weight(_text_:22 in 2587) [ClassicSimilarity], result of:
0.03400239 = score(doc=2587,freq=2.0), product of:
0.17576782 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05019314 = queryNorm
0.19345059 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Date
- 20. 1.2015 18:30:22
-
Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019)
0.01
0.0056670653 = product of:
0.017001195 = sum of:
0.017001195 = product of:
0.03400239 = sum of:
0.03400239 = weight(_text_:22 in 5287) [ClassicSimilarity], result of:
0.03400239 = score(doc=5287,freq=2.0), product of:
0.17576782 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.05019314 = queryNorm
0.19345059 = fieldWeight in 5287, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=5287)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Date
- 20. 1.2015 18:30:22
