-
Agata, T.: ¬A measure for evaluating search engines on the World Wide Web : retrieval test with ESL (Expected Search Length) (1997)
0.07
0.07235573 = product of:
0.18088932 = sum of:
0.11726888 = weight(_text_:wide in 3892) [ClassicSimilarity], result of:
0.11726888 = score(doc=3892,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.5874411 = fieldWeight in 3892, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.09375 = fieldNorm(doc=3892)
0.06362044 = weight(_text_:web in 3892) [ClassicSimilarity], result of:
0.06362044 = score(doc=3892,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.43268442 = fieldWeight in 3892, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.09375 = fieldNorm(doc=3892)
0.4 = coord(2/5)
-
Lazonder, A.W.; Biemans, H.J.A.; Wopereis, I.G.J.H.: Differences between novice and experienced users in searching information on the World Wide Web (2000)
0.05
0.051905703 = product of:
0.12976426 = sum of:
0.05863444 = weight(_text_:wide in 4598) [ClassicSimilarity], result of:
0.05863444 = score(doc=4598,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.29372054 = fieldWeight in 4598, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.046875 = fieldNorm(doc=4598)
0.07112982 = weight(_text_:web in 4598) [ClassicSimilarity], result of:
0.07112982 = score(doc=4598,freq=10.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.48375595 = fieldWeight in 4598, product of:
3.1622777 = tf(freq=10.0), with freq of:
10.0 = termFreq=10.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=4598)
0.4 = coord(2/5)
- Abstract
- Searching for information on the WWW basically comes down to locating an appropriate Web site and to retrieving relevant information from that site. This study examined the effect of a user's WWW experience on both phases of the search process. 35 students from 2 schools for Dutch pre-university education were observed while performing 3 search tasks. The results indicate that subjects with WWW-experience are more proficient in locating Web sites than are novice WWW-users. The observed differences were ascribed to the experts' superior skills in operating Web search engines. However, on tasks that required subjects to locate information on specific Web sites, the performance of experienced and novice users was equivalent - a result that is in line with hypertext research. Based on these findings, implications for training and supporting students in searching for information on the WWW are identified. Finally, the role of the subjects' level of domain expertise is discussed and directions for future research are proposed
-
Wu, C.-J.: Experiments on using the Dublin Core to reduce the retrieval error ratio (1998)
0.04
0.04220751 = product of:
0.10551877 = sum of:
0.06840685 = weight(_text_:wide in 5201) [ClassicSimilarity], result of:
0.06840685 = score(doc=5201,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.342674 = fieldWeight in 5201, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.0546875 = fieldNorm(doc=5201)
0.037111927 = weight(_text_:web in 5201) [ClassicSimilarity], result of:
0.037111927 = score(doc=5201,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.25239927 = fieldWeight in 5201, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0546875 = fieldNorm(doc=5201)
0.4 = coord(2/5)
- Abstract
- In order to test the power of metadata on information retrieval, an experiment was designed and conducted on a group of 7 graduate students using the Dublin Core as the cataloguing metadata. Results show that, on average, the retrieval error rate is only 2.9 per cent for the MES system (http://140.136.85.194), which utilizes the Dublin Core to describe the documents on the World Wide Web, in contrast to 20.7 per cent for the 7 famous search engines including HOTBOT, GAIS, LYCOS, EXCITE, INFOSEEK, YAHOO, and OCTOPUS. The very low error rate indicates that the users can use the information of the Dublin Core to decide whether to retrieve the documents or not
-
Khan, K.; Locatis, C.: Searching through cyberspace : the effects of link display and link density on information retrieval from hypertext on the World Wide Web (1998)
0.04
0.036177866 = product of:
0.09044466 = sum of:
0.05863444 = weight(_text_:wide in 446) [ClassicSimilarity], result of:
0.05863444 = score(doc=446,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.29372054 = fieldWeight in 446, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.046875 = fieldNorm(doc=446)
0.03181022 = weight(_text_:web in 446) [ClassicSimilarity], result of:
0.03181022 = score(doc=446,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.21634221 = fieldWeight in 446, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=446)
0.4 = coord(2/5)
-
Griesbaum, J.: Evaluierung hybrider Suchsysteme im WWW (2000)
0.04
0.036177866 = product of:
0.09044466 = sum of:
0.05863444 = weight(_text_:wide in 2482) [ClassicSimilarity], result of:
0.05863444 = score(doc=2482,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.29372054 = fieldWeight in 2482, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.046875 = fieldNorm(doc=2482)
0.03181022 = weight(_text_:web in 2482) [ClassicSimilarity], result of:
0.03181022 = score(doc=2482,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.21634221 = fieldWeight in 2482, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=2482)
0.4 = coord(2/5)
- Abstract
- Der Ausgangspunkt dieser Arbeit ist die Suchproblematik im World Wide Web. Suchmaschinen sind einerseits unverzichtbar für erfolgreiches Information Retrieval, andererseits wird ihnen eine mäßige Leistungsfähigkeit vorgeworfen. Das Thema dieser Arbeit ist die Untersuchung der Retrievaleffektivität deutschsprachiger Suchmaschinen. Es soll festgestellt werden, welche Retrievaleffektivität Nutzer derzeit erwarten können. Ein Ansatz, um die Retrievaleffektivität von Suchmaschinen zu erhöhen besteht darin, redaktionell von Menschen erstellte und automatisch generierte Suchergebnisse in einer Trefferliste zu vermengen. Ziel dieser Arbeit ist es, die Retrievaleffektivität solcher hybrider Systeme im Vergleich zu rein roboterbasierten Suchmaschinen zu evaluieren. Zunächst werden hierzu die grundlegenden Problembereiche bei der Evaluation von Retrievalsystemen analysiert. In Anlehnung an die von Tague-Sutcliff vorgeschlagene Methodik wird unter Beachtung der webspezifischen Besonderheiten eine mögliche Vorgehensweise erschlossen. Darauf aufbauend wird das konkrete Setting für die Durchführung der Evaluation erarbeitet und ein Retrievaleffektivitätstest bei den Suchmaschinen Lycos.de, AItaVista.de und QualiGo durchgeführt.
-
Dresel, R.; Hörnig, D.; Kaluza, H.; Peter, A.; Roßmann, A.; Sieber, W.: Evaluation deutscher Web-Suchwerkzeuge : Ein vergleichender Retrievaltest (2001)
0.03
0.033759642 = product of:
0.084399104 = sum of:
0.059981927 = weight(_text_:web in 261) [ClassicSimilarity], result of:
0.059981927 = score(doc=261,freq=4.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.4079388 = fieldWeight in 261, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0625 = fieldNorm(doc=261)
0.024417175 = product of:
0.04883435 = sum of:
0.04883435 = weight(_text_:22 in 261) [ClassicSimilarity], result of:
0.04883435 = score(doc=261,freq=2.0), product of:
0.15777399 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04505473 = queryNorm
0.30952093 = fieldWeight in 261, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=261)
0.5 = coord(1/2)
0.4 = coord(2/5)
- Abstract
- Die deutschen Suchmaschinen, Abacho, Acoon, Fireball und Lycos sowie die Web-Kataloge Web.de und Yahoo! werden einem Qualitätstest nach relativem Recall, Precision und Availability unterzogen. Die Methoden der Retrievaltests werden vorgestellt. Im Durchschnitt werden bei einem Cut-Off-Wert von 25 ein Recall von rund 22%, eine Precision von knapp 19% und eine Verfügbarkeit von 24% erreicht
-
Kantor, P.; Kim, M.H.; Ibraev, U.; Atasoy, K.: Estimating the number of relevant documents in enormous collections (1999)
0.03
0.030148221 = product of:
0.07537055 = sum of:
0.048862036 = weight(_text_:wide in 6690) [ClassicSimilarity], result of:
0.048862036 = score(doc=6690,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.24476713 = fieldWeight in 6690, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.0390625 = fieldNorm(doc=6690)
0.026508518 = weight(_text_:web in 6690) [ClassicSimilarity], result of:
0.026508518 = score(doc=6690,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.18028519 = fieldWeight in 6690, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0390625 = fieldNorm(doc=6690)
0.4 = coord(2/5)
- Abstract
- In assessing information retrieval systems, it is important to know not only the precision of the retrieved set, but also to compare the number of retrieved relevant items to the total number of relevant items. For large collections, such as the TREC test collections, or the World Wide Web, it is not possible to enumerate the entire set of relevant documents. If the retrieved documents are evaluated, a variant of the statistical "capture-recapture" method can be used to estimate the total number of relevant documents, providing the several retrieval systems used are sufficiently independent. We show that the underlying signal detection model supporting such an analysis can be extended in two ways. First, assuming that there are two distinct performance characteristics (corresponding to the chance of retrieving a relevant, and retrieving a given non-relevant document), we show that if there are three or more independent systems available it is possible to estimate the number of relevant documents without actually having to decide whether each individual document is relevant. We report applications of this 3-system method to the TREC data, leading to the conclusion that the independence assumptions are not satisfied. We then extend the model to a multi-system, multi-problem model, and show that it is possible to include statistical dependencies of all orders in the model, and determine the number of relevant documents for each of the problems in the set. Application to the TREC setting will be presented
-
Pemberton, J.K.; Ojala, M.; Garman, N.: Head to head : searching the Web versus traditional services (1998)
0.03
0.02673232 = product of:
0.0668308 = sum of:
0.042413626 = weight(_text_:web in 3572) [ClassicSimilarity], result of:
0.042413626 = score(doc=3572,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.2884563 = fieldWeight in 3572, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0625 = fieldNorm(doc=3572)
0.024417175 = product of:
0.04883435 = sum of:
0.04883435 = weight(_text_:22 in 3572) [ClassicSimilarity], result of:
0.04883435 = score(doc=3572,freq=2.0), product of:
0.15777399 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04505473 = queryNorm
0.30952093 = fieldWeight in 3572, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=3572)
0.5 = coord(1/2)
0.4 = coord(2/5)
- Source
- Online. 22(1998) no.3, S.24-26,28
-
¬The Eleventh Text Retrieval Conference, TREC 2002 (2003)
0.03
0.02673232 = product of:
0.0668308 = sum of:
0.042413626 = weight(_text_:web in 4049) [ClassicSimilarity], result of:
0.042413626 = score(doc=4049,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.2884563 = fieldWeight in 4049, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0625 = fieldNorm(doc=4049)
0.024417175 = product of:
0.04883435 = sum of:
0.04883435 = weight(_text_:22 in 4049) [ClassicSimilarity], result of:
0.04883435 = score(doc=4049,freq=2.0), product of:
0.15777399 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04505473 = queryNorm
0.30952093 = fieldWeight in 4049, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=4049)
0.5 = coord(1/2)
0.4 = coord(2/5)
- Abstract
- Proceedings of the llth TREC-conference held in Gaithersburg, Maryland (USA), November 19-22, 2002. Aim of the conference was discussion an retrieval and related information-seeking tasks for large test collection. 93 research groups used different techniques, for information retrieval from the same large database. This procedure makes it possible to compare the results. The tasks are: Cross-language searching, filtering, interactive searching, searching for novelty, question answering, searching for video shots, and Web searching.
-
Voorhees, E.M.; Harman, D.K.: ¬The Text REtrieval Conference (2005)
0.02
0.021103755 = product of:
0.052759387 = sum of:
0.034203425 = weight(_text_:wide in 5082) [ClassicSimilarity], result of:
0.034203425 = score(doc=5082,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.171337 = fieldWeight in 5082, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.02734375 = fieldNorm(doc=5082)
0.018555963 = weight(_text_:web in 5082) [ClassicSimilarity], result of:
0.018555963 = score(doc=5082,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.12619963 = fieldWeight in 5082, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.02734375 = fieldNorm(doc=5082)
0.4 = coord(2/5)
- Abstract
- Text retrieval technology targets a problem that is all too familiar: finding relevant information in large stores of electronic documents. The problem is an old one, with the first research conference devoted to the subject held in 1958 [11]. Since then the problem has continued to grow as more information is created in electronic form and more people gain electronic access. The advent of the World Wide Web, where anyone can publish so everyone must search, is a graphic illustration of the need for effective retrieval technology. The Text REtrieval Conference (TREC) is a workshop series designed to build the infrastructure necessary for the large-scale evaluation of text retrieval technology, thereby accelerating its transfer into the commercial sector. The series is sponsored by the U.S. National Institute of Standards and Technology (NIST) and the U.S. Department of Defense. At the time of this writing, there have been twelve TREC workshops and preparations for the thirteenth workshop are under way. Participants in the workshops have been drawn from the academic, commercial, and government sectors, and have included representatives from more than twenty different countries. These collective efforts have accomplished a great deal: a variety of large test collections have been built for both traditional ad hoc retrieval and related tasks such as cross-language retrieval, speech retrieval, and question answering; retrieval effectiveness has approximately doubled; and many commercial retrieval systems now contain technology first developed in TREC.
-
Bar-Ilan, J.: ¬The Web as an information source on informetrics? : A content analysis (2000)
0.02
0.017994579 = product of:
0.08997289 = sum of:
0.08997289 = weight(_text_:web in 4587) [ClassicSimilarity], result of:
0.08997289 = score(doc=4587,freq=16.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.6119082 = fieldWeight in 4587, product of:
4.0 = tf(freq=16.0), with freq of:
16.0 = termFreq=16.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=4587)
0.2 = coord(1/5)
- Abstract
- This article addresses the question of whether the Web can serve as an information source for research. Specifically, it analyzes by way of content analysis the Web pages retrieved by the major search engines on a particular date (June 7, 1998), as a result of the query 'informetrics OR informetric'. In 807 out of the 942 retrieved pages, the search terms were mentioned in the context of information science. Over 70% of the pages contained only indirect information on the topic, in the form of hypertext links and bibliographical references without annotation. The bibliographical references extracted from the Web pages were analyzed, and lists of most productive authors, most cited authors, works, and sources were compiled. The list of reference obtained from the Web was also compared to data retrieved from commercial databases. For most cases, the list of references extracted from the Web outperformed the commercial, bibliographic databases. The results of these comparisons indicate that valuable, freely available data is hidden in the Web waiting to be extracted from the millions of Web pages
-
Clarke, S.J.; Willett, P.: Estimating the recall performance of Web search engines (1997)
0.02
0.01696545 = product of:
0.08482725 = sum of:
0.08482725 = weight(_text_:web in 760) [ClassicSimilarity], result of:
0.08482725 = score(doc=760,freq=8.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.5769126 = fieldWeight in 760, product of:
2.828427 = tf(freq=8.0), with freq of:
8.0 = termFreq=8.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0625 = fieldNorm(doc=760)
0.2 = coord(1/5)
- Abstract
- Reports a comparison of the retrieval effectiveness of the AltaVista, Excite and Lycos Web search engines. Describes a method for comparing the recall of the 3 sets of searches, despite the fact that they are carried out on non identical sets of Web pages. It is thus possible, unlike previous comparative studies of Web search engines, to consider both recall and precision when evaluating the effectiveness of search engines
-
MacFarlane, A.: Evaluation of web search for the information practitioner (2007)
0.02
0.016832387 = product of:
0.08416194 = sum of:
0.08416194 = weight(_text_:web in 817) [ClassicSimilarity], result of:
0.08416194 = score(doc=817,freq=14.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.57238775 = fieldWeight in 817, product of:
3.7416575 = tf(freq=14.0), with freq of:
14.0 = termFreq=14.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=817)
0.2 = coord(1/5)
- Abstract
- Purpose - The aim of the paper is to put forward a structured mechanism for web search evaluation. The paper seeks to point to useful scientific research and show how information practitioners can use these methods in evaluation of search on the web for their users. Design/methodology/approach - The paper puts forward an approach which utilizes traditional laboratory-based evaluation measures such as average precision/precision at N documents, augmented with diagnostic measures such as link broken, etc., which are used to show why precision measures are depressed as well as the quality of the search engines crawling mechanism. Findings - The paper shows how to use diagnostic measures in conjunction with precision in order to evaluate web search. Practical implications - The methodology presented in this paper will be useful to any information professional who regularly uses web search as part of their information seeking and needs to evaluate web search services. Originality/value - The paper argues that the use of diagnostic measures is essential in web search, as precision measures on their own do not allow a searcher to understand why search results differ between search engines.
-
Ravana, S.D.; Taheri, M.S.; Rajagopal, P.: Document-based approach to improve the accuracy of pairwise comparison in evaluating information retrieval systems (2015)
0.02
0.016707702 = product of:
0.04176925 = sum of:
0.026508518 = weight(_text_:web in 2587) [ClassicSimilarity], result of:
0.026508518 = score(doc=2587,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.18028519 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.015260735 = product of:
0.03052147 = sum of:
0.03052147 = weight(_text_:22 in 2587) [ClassicSimilarity], result of:
0.03052147 = score(doc=2587,freq=2.0), product of:
0.15777399 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04505473 = queryNorm
0.19345059 = fieldWeight in 2587, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=2587)
0.5 = coord(1/2)
0.4 = coord(2/5)
- Abstract
- Purpose The purpose of this paper is to propose a method to have more accurate results in comparing performance of the paired information retrieval (IR) systems with reference to the current method, which is based on the mean effectiveness scores of the systems across a set of identified topics/queries. Design/methodology/approach Based on the proposed approach, instead of the classic method of using a set of topic scores, the documents level scores are considered as the evaluation unit. These document scores are the defined document's weight, which play the role of the mean average precision (MAP) score of the systems as a significance test's statics. The experiments were conducted using the TREC 9 Web track collection. Findings The p-values generated through the two types of significance tests, namely the Student's t-test and Mann-Whitney show that by using the document level scores as an evaluation unit, the difference between IR systems is more significant compared with utilizing topic scores. Originality/value Utilizing a suitable test collection is a primary prerequisite for IR systems comparative evaluation. However, in addition to reusable test collections, having an accurate statistical testing is a necessity for these evaluations. The findings of this study will assist IR researchers to evaluate their retrieval systems and algorithms more accurately.
- Date
- 20. 1.2015 18:30:22
-
Frei, H.P.; Meienberg, S.; Schäuble, P.: ¬The perils of interpreting recall and precision values (1991)
0.02
0.015635852 = product of:
0.078179255 = sum of:
0.078179255 = weight(_text_:wide in 786) [ClassicSimilarity], result of:
0.078179255 = score(doc=786,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.3916274 = fieldWeight in 786, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.0625 = fieldNorm(doc=786)
0.2 = coord(1/5)
- Abstract
- The traditional recall and precision measure is inappropriate when retrieval algorithms that retrieve information from Wide Area Networks are evaluated. The principle reason is that information available in WANs is dynamic and its size os orders of magnitude greater than the size of the usual test collections. To overcome these problems, a new efffectiveness measure has been developed, which we call the 'usefulness measure'
-
Madelung, H.-O.: Subject searching in the social sciences : a comparison of PRECIS and KWIC indexes indexes to newspaper articles (1982)
0.01
0.01368137 = product of:
0.06840685 = sum of:
0.06840685 = weight(_text_:wide in 5517) [ClassicSimilarity], result of:
0.06840685 = score(doc=5517,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.342674 = fieldWeight in 5517, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.0546875 = fieldNorm(doc=5517)
0.2 = coord(1/5)
- Abstract
- 89 articles from a small, Danish left-wing newspaper were indexed by PRECIS and KWIC. The articles cover a wide range of social science subjects. Controlled test searches in both indexes were carried out by 20 students of library science. The results obtained from this small-scale retrieval test were evaluated by a chi-square test. The PRECIS index led to more correct answers and fewer wrong answers than the KWIC index, i.e. it had both better recall and greater precision. Furthermore, the students were more confident in their judgement of the relevance of retrieved articles in the PRECIS index than in the KWIC index; and they generally favoured the PRECIS index in the subjective judgement they were asked to make
-
Harman, D.: ¬The Text REtrieval Conferences (TRECs) : providing a test-bed for information retrieval systems (1998)
0.01
0.01368137 = product of:
0.06840685 = sum of:
0.06840685 = weight(_text_:wide in 1314) [ClassicSimilarity], result of:
0.06840685 = score(doc=1314,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.342674 = fieldWeight in 1314, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.0546875 = fieldNorm(doc=1314)
0.2 = coord(1/5)
- Abstract
- The Text REtrieval Conference (TREC) workshop series encourages research in information retrieval from large text applications by providing a large test collection, uniform scoring procedures and a forum for organizations interested in comparing their results. Now in its seventh year, the conference has become the major experimental effort in the field. Participants in the TREC conferences have examined a wide variety of retrieval techniques, including methods using automatic thesauri, sophisticated term weighting, natural language techniques, relevance feedback and advanced pattern matching. The TREC conference series is co-sponsored by the National Institute of Standards and Technology (NIST) and the Information Technology Office of the Defense Advanced Research Projects Agency (DARPA)
-
Gilchrist, A.: Research and consultancy (1998)
0.01
0.013020389 = product of:
0.065101944 = sum of:
0.065101944 = product of:
0.13020389 = sum of:
0.13020389 = weight(_text_:server in 1394) [ClassicSimilarity], result of:
0.13020389 = score(doc=1394,freq=2.0), product of:
0.25762302 = queryWeight, product of:
5.7180014 = idf(docFreq=394, maxDocs=44218)
0.04505473 = queryNorm
0.5054047 = fieldWeight in 1394, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.7180014 = idf(docFreq=394, maxDocs=44218)
0.0625 = fieldNorm(doc=1394)
0.5 = coord(1/2)
0.2 = coord(1/5)
- Abstract
- State of the art review of literature published about research and consultancy in library and information science (LIS). Issues covered include: scope and definitions of what constitutes research and consultancy; funding of research and development; national LIS research and the funding agencies; electronic libraries; document delivery; multimedia document delivery; the Z39.50 standard for client server computer architecture, the Internet and WWW; electronic publishing; information retrieval; evaluation and evaluation techniques; the Text Retrieval Conferences (TREC); the user domain; management issues; decision support systems; information politics and organizational culture; and value for money issues
-
Hawking, D.; Craswell, N.: ¬The very large collection and Web tracks (2005)
0.01
0.0127240885 = product of:
0.06362044 = sum of:
0.06362044 = weight(_text_:web in 5085) [ClassicSimilarity], result of:
0.06362044 = score(doc=5085,freq=2.0), product of:
0.14703658 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.04505473 = queryNorm
0.43268442 = fieldWeight in 5085, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.09375 = fieldNorm(doc=5085)
0.2 = coord(1/5)
-
Schabas, A.H.: Postcoordinate retrieval : a comparison of two retrieval languages (1982)
0.01
0.011726889 = product of:
0.05863444 = sum of:
0.05863444 = weight(_text_:wide in 1202) [ClassicSimilarity], result of:
0.05863444 = score(doc=1202,freq=2.0), product of:
0.19962662 = queryWeight, product of:
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.04505473 = queryNorm
0.29372054 = fieldWeight in 1202, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.4307585 = idf(docFreq=1430, maxDocs=44218)
0.046875 = fieldNorm(doc=1202)
0.2 = coord(1/5)
- Abstract
- This article reports on a comparison of the postcoordinate retrieval effectiveness of two indexing languages: LCSH and PRECIS. The effect of augmenting each with title words was also studies. The database for the study was over 15.000 UK MARC records. Users returned 5.326 relevant judgements for citations retrieved for 61 SDI profiles, representing a wide variety of subjects. Results are reported in terms of precision and relative recall. Pure/applied sciences data and social science data were analyzed separately. Cochran's significance tests for ratios were used to interpret the findings. Recall emerged as the more important measure discriminating the behavior of the two languages. Addition of title words was found to improve recall of both indexing languages significantly. A direct relationship was observed between recall and exhaustivity. For the social sciences searches, recalls from PRECIS alone and from PRECIS with title words were significantly higher than those from LCSH alone and from LCSH with title words, respectively. Corresponding comparisons for the pure/applied sciences searches revealed no significant differences
