-
Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017)
0.05
0.04686443 = product of:
0.09372886 = sum of:
0.043081827 = weight(_text_:digital in 3700) [ClassicSimilarity], result of:
0.043081827 = score(doc=3700,freq=2.0), product of:
0.19770671 = queryWeight, product of:
3.944552 = idf(docFreq=2326, maxDocs=44218)
0.050121464 = queryNorm
0.21790776 = fieldWeight in 3700, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.944552 = idf(docFreq=2326, maxDocs=44218)
0.0390625 = fieldNorm(doc=3700)
0.050647035 = weight(_text_:library in 3700) [ClassicSimilarity], result of:
0.050647035 = score(doc=3700,freq=14.0), product of:
0.1317883 = queryWeight, product of:
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.050121464 = queryNorm
0.384306 = fieldWeight in 3700, product of:
3.7416575 = tf(freq=14.0), with freq of:
14.0 = termFreq=14.0
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.0390625 = fieldNorm(doc=3700)
0.5 = coord(2/4)
- Abstract
- Purpose This paper demonstrates how to apply traditional information retrieval evaluation methods based on standards from the Text REtrieval Conference (TREC) and web search evaluation to all types of modern library information systems including online public access catalogs, discovery systems, and digital libraries that provide web search features to gather information from heterogeneous sources. Design/methodology/approach We apply conventional procedures from information retrieval evaluation to the library information system context considering the specific characteristics of modern library materials. Findings We introduce a framework consisting of five parts: (1) search queries, (2) search results, (3) assessors, (4) testing, and (5) data analysis. We show how to deal with comparability problems resulting from diverse document types, e.g., electronic articles vs. printed monographs and what issues need to be considered for retrieval tests in the library context. Practical implications The framework can be used as a guideline for conducting retrieval effectiveness studies in the library context. Originality/value Although a considerable amount of research has been done on information retrieval evaluation, and standards for conducting retrieval effectiveness studies do exist, to our knowledge this is the first attempt to provide a systematic framework for evaluating the retrieval effectiveness of twenty-first-century library information systems. We demonstrate which issues must be considered and what decisions must be made by researchers prior to a retrieval test.
-
Munkelt, J.; Schaer, P.: Towards an IR test collection for the German National Library (2018)
0.01
0.013399946 = product of:
0.053599782 = sum of:
0.053599782 = weight(_text_:library in 5780) [ClassicSimilarity], result of:
0.053599782 = score(doc=5780,freq=2.0), product of:
0.1317883 = queryWeight, product of:
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.050121464 = queryNorm
0.40671125 = fieldWeight in 5780, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.109375 = fieldNorm(doc=5780)
0.25 = coord(1/4)
-
Munkelt, J.; Schaer, P.; Lepsky, K.: Towards an IR test collection for the German National Library (2018)
0.01
0.009946881 = product of:
0.039787523 = sum of:
0.039787523 = weight(_text_:library in 4311) [ClassicSimilarity], result of:
0.039787523 = score(doc=4311,freq=6.0), product of:
0.1317883 = queryWeight, product of:
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.050121464 = queryNorm
0.30190483 = fieldWeight in 4311, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.046875 = fieldNorm(doc=4311)
0.25 = coord(1/4)
- Abstract
- Automatic content indexing is one of the innovations that are increasingly changing the way libraries work. In theory, it promises a cataloguing service that would hardly be possible with humans in terms of speed, quantity and maybe quality. The German National Library (DNB) has also recognised this potential and is increasingly relying on the automatic indexing of their catalogue content. The DNB took a major step in this direction in 2017, which was announced in two papers. The announcement was rather restrained, but the content of the papers is all the more explosive for the library community: Since September 2017, the DNB has discontinued the intellectual indexing of series Band H and has switched to an automatic process for these series. The subject indexing of online publications (series O) has been purely automatical since 2010; from September 2017, monographs and periodicals published outside the publishing industry and university publications will no longer be indexed by people. This raises the question: What is the quality of the automatic indexing compared to the manual work or in other words to which degree can the automatic indexing replace people without a signi cant drop in regards to quality?
-
Schirrmeister, N.-P.; Keil, S.: Aufbau einer Infrastruktur für Information Retrieval-Evaluationen (2012)
0.01
0.009866329 = product of:
0.039465316 = sum of:
0.039465316 = product of:
0.07893063 = sum of:
0.07893063 = weight(_text_:project in 3097) [ClassicSimilarity], result of:
0.07893063 = score(doc=3097,freq=2.0), product of:
0.21156175 = queryWeight, product of:
4.220981 = idf(docFreq=1764, maxDocs=44218)
0.050121464 = queryNorm
0.37308553 = fieldWeight in 3097, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.220981 = idf(docFreq=1764, maxDocs=44218)
0.0625 = fieldNorm(doc=3097)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- Das Projekt "Aufbau einer Infrastruktur für Information Retrieval-Evaluationen" (AIIRE) bietet eine Softwareinfrastruktur zur Unterstützung von Information Retrieval-Evaluationen (IR-Evaluationen). Die Infrastruktur basiert auf einem Tool-Kit, das bei GESIS im Rahmen des DFG-Projekts IRM entwickelt wurde. Ziel ist es, ein System zu bieten, das zur Forschung und Lehre am Fachbereich Media für IR-Evaluationen genutzt werden kann. This paper describes some aspects of a project called "Aufbau einer Infrastruktur für Information Retrieval-Evaluationen" (AIIRE). Its goal is to build a software-infrastructure which supports the evaluation of information retrieval algorithms.
-
Hider, P.: ¬The search value added by professional indexing to a bibliographic database (2017)
0.01
0.0067679947 = product of:
0.027071979 = sum of:
0.027071979 = weight(_text_:library in 3868) [ClassicSimilarity], result of:
0.027071979 = score(doc=3868,freq=4.0), product of:
0.1317883 = queryWeight, product of:
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.050121464 = queryNorm
0.2054202 = fieldWeight in 3868, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.0390625 = fieldNorm(doc=3868)
0.25 = coord(1/4)
- Abstract
- Gross et al. (2015) have demonstrated that about a quarter of hits would typically be lost to keyword searchers if contemporary academic library catalogs dropped their controlled subject headings. This paper reports on an analysis of the loss levels that would result if a bibliographic database, namely the Australian Education Index (AEI), were missing the subject descriptors and identifiers assigned by its professional indexers, employing the methodology developed by Gross and Taylor (2005), and later by Gross et al. (2015). The results indicate that AEI users would lose a similar proportion of hits per query to that experienced by library catalog users: on average, 27% of the resources found by a sample of keyword queries on the AEI database would not have been found without the subject indexing, based on the Australian Thesaurus of Education Descriptors (ATED). The paper also discusses the methodological limitations of these studies, pointing out that real-life users might still find some of the resources missed by a particular query through follow-up searches, while additional resources might also be found through iterative searching on the subject vocabulary. The paper goes on to describe a new research design, based on a before - and - after experiment, which addresses some of these limitations. It is argued that this alternative design will provide a more realistic picture of the value that professionally assigned subject indexing and controlled subject vocabularies can add to literature searching of a more scholarly and thorough kind.
-
Reichert, S.; Mayr, P.: Untersuchung von Relevanzeigenschaften in einem kontrollierten Eyetracking-Experiment (2012)
0.01
0.005093075 = product of:
0.0203723 = sum of:
0.0203723 = product of:
0.0407446 = sum of:
0.0407446 = weight(_text_:22 in 328) [ClassicSimilarity], result of:
0.0407446 = score(doc=328,freq=2.0), product of:
0.17551683 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.050121464 = queryNorm
0.23214069 = fieldWeight in 328, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=328)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 22. 7.2012 19:25:54
-
Leiva-Mederos, A.; Senso, J.A.; Hidalgo-Delgado, Y.; Hipola, P.: Working framework of semantic interoperability for CRIS with heterogeneous data sources (2017)
0.00
0.0049331645 = product of:
0.019732658 = sum of:
0.019732658 = product of:
0.039465316 = sum of:
0.039465316 = weight(_text_:project in 3706) [ClassicSimilarity], result of:
0.039465316 = score(doc=3706,freq=2.0), product of:
0.21156175 = queryWeight, product of:
4.220981 = idf(docFreq=1764, maxDocs=44218)
0.050121464 = queryNorm
0.18654276 = fieldWeight in 3706, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.220981 = idf(docFreq=1764, maxDocs=44218)
0.03125 = fieldNorm(doc=3706)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- Purpose Information from Current Research Information Systems (CRIS) is stored in different formats, in platforms that are not compatible, or even in independent networks. It would be helpful to have a well-defined methodology to allow for management data processing from a single site, so as to take advantage of the capacity to link disperse data found in different systems, platforms, sources and/or formats. Based on functionalities and materials of the VLIR project, the purpose of this paper is to present a model that provides for interoperability by means of semantic alignment techniques and metadata crosswalks, and facilitates the fusion of information stored in diverse sources. Design/methodology/approach After reviewing the state of the art regarding the diverse mechanisms for achieving semantic interoperability, the paper analyzes the following: the specific coverage of the data sets (type of data, thematic coverage and geographic coverage); the technical specifications needed to retrieve and analyze a distribution of the data set (format, protocol, etc.); the conditions of re-utilization (copyright and licenses); and the "dimensions" included in the data set as well as the semantics of these dimensions (the syntax and the taxonomies of reference). The semantic interoperability framework here presented implements semantic alignment and metadata crosswalk to convert information from three different systems (ABCD, Moodle and DSpace) to integrate all the databases in a single RDF file. Findings The paper also includes an evaluation based on the comparison - by means of calculations of recall and precision - of the proposed model and identical consultations made on Open Archives Initiative and SQL, in order to estimate its efficiency. The results have been satisfactory enough, due to the fact that the semantic interoperability facilitates the exact retrieval of information. Originality/value The proposed model enhances management of the syntactic and semantic interoperability of the CRIS system designed. In a real setting of use it achieves very positive results.
-
Hider, P.: ¬The search value added by professional indexing to a bibliographic database (2018)
0.00
0.004785695 = product of:
0.01914278 = sum of:
0.01914278 = weight(_text_:library in 4300) [ClassicSimilarity], result of:
0.01914278 = score(doc=4300,freq=2.0), product of:
0.1317883 = queryWeight, product of:
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.050121464 = queryNorm
0.14525402 = fieldWeight in 4300, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.0390625 = fieldNorm(doc=4300)
0.25 = coord(1/4)
- Abstract
- Gross et al. (2015) have demonstrated that about a quarter of hits would typically be lost to keyword searchers if contemporary academic library catalogs dropped their controlled subject headings. This article reports on an investigation of the search value that subject descriptors and identifiers assigned by professional indexers add to a bibliographic database, namely the Australian Education Index (AEI). First, a similar methodology to that developed by Gross et al. (2015) was applied, with keyword searches representing a range of educational topics run on the AEI database with and without its subject indexing. The results indicated that AEI users would also lose, on average, about a quarter of hits per query. Second, an alternative research design was applied in which an experienced literature searcher was asked to find resources on a set of educational topics on an AEI database stripped of its subject indexing and then asked to search for additional resources on the same topics after the subject indexing had been reinserted. In this study, the proportion of additional resources that would have been lost had it not been for the subject indexing was again found to be about a quarter of the total resources found for each topic, on average.
-
Pal, S.; Mitra, M.; Kamps, J.: Evaluation effort, reliability and reusability in XML retrieval (2011)
0.00
0.0042442293 = product of:
0.016976917 = sum of:
0.016976917 = product of:
0.033953834 = sum of:
0.033953834 = weight(_text_:22 in 4197) [ClassicSimilarity], result of:
0.033953834 = score(doc=4197,freq=2.0), product of:
0.17551683 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.050121464 = queryNorm
0.19345059 = fieldWeight in 4197, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4197)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 22. 1.2011 14:20:56
-
Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011)
0.00
0.0042442293 = product of:
0.016976917 = sum of:
0.016976917 = product of:
0.033953834 = sum of:
0.033953834 = weight(_text_:22 in 4540) [ClassicSimilarity], result of:
0.033953834 = score(doc=4540,freq=2.0), product of:
0.17551683 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.050121464 = queryNorm
0.19345059 = fieldWeight in 4540, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=4540)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 12. 7.2011 18:29:22
-
Wildemuth, B.; Freund, L.; Toms, E.G.: Untangling search task complexity and difficulty in the context of interactive information retrieval studies (2014)
0.00
0.0042442293 = product of:
0.016976917 = sum of:
0.016976917 = product of:
0.033953834 = sum of:
0.033953834 = weight(_text_:22 in 1786) [ClassicSimilarity], result of:
0.033953834 = score(doc=1786,freq=2.0), product of:
0.17551683 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.050121464 = queryNorm
0.19345059 = fieldWeight in 1786, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=1786)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 6. 4.2015 19:31:22
-
Ravana, S.D.; Taheri, M.S.; Rajagopal, P.: Document-based approach to improve the accuracy of pairwise comparison in evaluating information retrieval systems (2015)
0.00
0.0042442293 = product of:
0.016976917 = sum of:
0.016976917 = product of:
0.033953834 = sum of:
0.033953834 = weight(_text_:22 in 2587) [ClassicSimilarity], result of:
0.033953834 = score(doc=2587,freq=2.0), product of:
0.17551683 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.050121464 = queryNorm
0.19345059 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 20. 1.2015 18:30:22
-
Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019)
0.00
0.0042442293 = product of:
0.016976917 = sum of:
0.016976917 = product of:
0.033953834 = sum of:
0.033953834 = weight(_text_:22 in 5287) [ClassicSimilarity], result of:
0.033953834 = score(doc=5287,freq=2.0), product of:
0.17551683 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.050121464 = queryNorm
0.19345059 = fieldWeight in 5287, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=5287)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 20. 1.2015 18:30:22
