-
Lewandowski, D.: Evaluating the retrieval effectiveness of web search engines using a representative query sample (2015)
0.03
0.027245669 = product of:
0.054491337 = sum of:
0.0070626684 = product of:
0.028250674 = sum of:
0.028250674 = weight(_text_:based in 2157) [ClassicSimilarity], result of:
0.028250674 = score(doc=2157,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.19973516 = fieldWeight in 2157, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.046875 = fieldNorm(doc=2157)
0.25 = coord(1/4)
0.047428668 = product of:
0.094857335 = sum of:
0.094857335 = weight(_text_:assessment in 2157) [ClassicSimilarity], result of:
0.094857335 = score(doc=2157,freq=2.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.36599535 = fieldWeight in 2157, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.046875 = fieldNorm(doc=2157)
0.5 = coord(1/2)
0.5 = coord(2/4)
- Abstract
- Search engine retrieval effectiveness studies are usually small scale, using only limited query samples. Furthermore, queries are selected by the researchers. We address these issues by taking a random representative sample of 1,000 informational and 1,000 navigational queries from a major German search engine and comparing Google's and Bing's results based on this sample. Jurors were found through crowdsourcing, and data were collected using specialized software, the Relevance Assessment Tool (RAT). We found that although Google outperforms Bing in both query types, the difference in the performance for informational queries was rather low. However, for navigational queries, Google found the correct answer in 95.3% of cases, whereas Bing only found the correct answer 76.6% of the time. We conclude that search engine performance on navigational queries is of great importance, because users in this case can clearly identify queries that have returned correct results. So, performance on this query type may contribute to explaining user satisfaction with search engines.
-
Lewandowski, D.; Sünkler, S.: ¬Das Relevance Assessment Tool : eine modulare Software zur Unterstützung bei der Durchführung vielfältiger Studien mit Suchmaschinen (2019)
0.02
0.01676857 = product of:
0.06707428 = sum of:
0.06707428 = product of:
0.13414855 = sum of:
0.13414855 = weight(_text_:assessment in 5026) [ClassicSimilarity], result of:
0.13414855 = score(doc=5026,freq=4.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.51759565 = fieldWeight in 5026, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.046875 = fieldNorm(doc=5026)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- In diesem Artikel stellen wir eine Software vor, mit der sich Studien zu Such- und Informationssystemen realisieren lassen. Das Relevance Assessment Tool (RAT) soll umfangreiche Untersuchungen mit Daten von kommerziellen Suchmaschinen unterstützen. Die Software ist modular und webbasiert. Es lassen sich damit automatisiert Daten von Suchmaschinen erfassen. Dazu können Studien mit Fragen und Skalen flexibel gestaltet und die Informationsobjekte anhand der Fragen durch Juroren bewertet werden. Durch die Modularität lassen sich die einzelnen Komponenten für eine Vielzahl von Studien nutzen, die sich auf Web-Inhalte beziehen. So kann die Software auch für qualitative Inhaltsanalysen eingesetzt werden oder durch das automatisierte Scraping eine große Datenbasis an Web-Dokumenten liefern, die sich quantitativ in empirischen Studien analysieren lassen.
-
Haring, M.; Rudaev, A.; Lewandowski, D.: Google & Co. : wie die "Search Studies" an der HAW Hamburg unserem Nutzungsverhalten auf den Zahn fühlen: Blickpunkt angewandte Forschung (2022)
0.02
0.015809556 = product of:
0.063238226 = sum of:
0.063238226 = product of:
0.12647645 = sum of:
0.12647645 = weight(_text_:assessment in 630) [ClassicSimilarity], result of:
0.12647645 = score(doc=630,freq=2.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.4879938 = fieldWeight in 630, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.0625 = fieldNorm(doc=630)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- Die Forschungsgruppe Search Studies forscht an der HAW Hamburg zur Nutzung kommerzieller Suchmaschinen, zur Suchmaschinenoptimierung und zum Relevance Assessment von Suchmaschinen. Der Leiter der Forschungsgruppe, Prof. Dr. Dirk Lewandowski, stand für ein Interview zu seiner Tätigkeit und der seines Teams, sowie seiner Lehre an der HAW Hamburg zur Verfügung. Sollten wir Informationen aus dem Internet vertrauen oder ist Vorsicht angebracht?
-
Schaer, P.; Mayr, P.; Sünkler, S.; Lewandowski, D.: How relevant is the long tail? : a relevance assessment study on million short (2016)
0.01
0.013973806 = product of:
0.055895224 = sum of:
0.055895224 = product of:
0.11179045 = sum of:
0.11179045 = weight(_text_:assessment in 3144) [ClassicSimilarity], result of:
0.11179045 = score(doc=3144,freq=4.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.43132967 = fieldWeight in 3144, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.0390625 = fieldNorm(doc=3144)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- Users of web search engines are known to mostly focus on the top ranked results of the search engine result page. While many studies support this well known information seeking pattern only few studies concentrate on the question what users are missing by neglecting lower ranked results. To learn more about the relevance distributions in the so-called long tail we conducted a relevance assessment study with the Million Short long-tail web search engine. While we see a clear difference in the content between the head and the tail of the search engine result list we see no statistical significant differences in the binary relevance judgments and weak significant differences when using graded relevance. The tail contains different but still valuable results. We argue that the long tail can be a rich source for the diversification of web search engine result lists but it needs more evaluation to clearly describe the differences.
-
Lewandowski, D.; Sünkler, S.: What does Google recommend when you want to compare insurance offerings? (2019)
0.01
0.010893034 = product of:
0.021786068 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 5288) [ClassicSimilarity], result of:
0.023542227 = score(doc=5288,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 5288, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=5288)
0.25 = coord(1/4)
0.015900511 = product of:
0.031801023 = sum of:
0.031801023 = weight(_text_:22 in 5288) [ClassicSimilarity], result of:
0.031801023 = score(doc=5288,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.19345059 = fieldWeight in 5288, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=5288)
0.5 = coord(1/2)
0.5 = coord(2/4)
- Abstract
- Purpose The purpose of this paper is to describe a new method to improve the analysis of search engine results by considering the provider level as well as the domain level. This approach is tested by conducting a study using queries on the topic of insurance comparisons. Design/methodology/approach The authors conducted an empirical study that analyses the results of search queries aimed at comparing insurance companies. The authors used a self-developed software system that automatically queries commercial search engines and automatically extracts the content of the returned result pages for further data analysis. The data analysis was carried out using the KNIME Analytics Platform. Findings Google's top search results are served by only a few providers that frequently appear in these results. The authors show that some providers operate several domains on the same topic and that these domains appear for the same queries in the result lists. Research limitations/implications The authors demonstrate the feasibility of this approach and draw conclusions for further investigations from the empirical study. However, the study is a limited use case based on a limited number of search queries. Originality/value The proposed method allows large-scale analysis of the composition of the top results from commercial search engines. It allows using valid empirical data to determine what users actually see on the search engine result pages.
- Date
- 20. 1.2015 18:30:22
-
Lewandowski, D.: Alles nur noch Google? : Entwicklungen im Bereich der WWW-Suchmaschinen (2002)
0.01
0.006360204 = product of:
0.025440816 = sum of:
0.025440816 = product of:
0.05088163 = sum of:
0.05088163 = weight(_text_:22 in 997) [ClassicSimilarity], result of:
0.05088163 = score(doc=997,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.30952093 = fieldWeight in 997, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=997)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 29. 9.2002 18:49:22
-
Lewandowski, D.: Abfragesprachen und erweiterte Funktionen von WWW-Suchmaschinen (2004)
0.01
0.006360204 = product of:
0.025440816 = sum of:
0.025440816 = product of:
0.05088163 = sum of:
0.05088163 = weight(_text_:22 in 2314) [ClassicSimilarity], result of:
0.05088163 = score(doc=2314,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.30952093 = fieldWeight in 2314, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=2314)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 28.11.2004 13:11:22
-
Lewandowski, D.: Query understanding (2011)
0.01
0.006360204 = product of:
0.025440816 = sum of:
0.025440816 = product of:
0.05088163 = sum of:
0.05088163 = weight(_text_:22 in 344) [ClassicSimilarity], result of:
0.05088163 = score(doc=344,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.30952093 = fieldWeight in 344, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=344)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 18. 9.2018 18:22:18
-
Lewandowski, D.: ¬Die Macht der Suchmaschinen und ihr Einfluss auf unsere Entscheidungen (2014)
0.00
0.0047701527 = product of:
0.019080611 = sum of:
0.019080611 = product of:
0.038161222 = sum of:
0.038161222 = weight(_text_:22 in 1491) [ClassicSimilarity], result of:
0.038161222 = score(doc=1491,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.23214069 = fieldWeight in 1491, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=1491)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 22. 9.2014 18:54:11
-
Lewandowski, D.; Spree, U.: Ranking of Wikipedia articles in search engines revisited : fair ranking for reasonable quality? (2011)
0.00
0.003975128 = product of:
0.015900511 = sum of:
0.015900511 = product of:
0.031801023 = sum of:
0.031801023 = weight(_text_:22 in 444) [ClassicSimilarity], result of:
0.031801023 = score(doc=444,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.19345059 = fieldWeight in 444, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=444)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 30. 9.2012 19:27:22
-
Lewandowski, D.: How can library materials be ranked in the OPAC? (2009)
0.00
0.0020808585 = product of:
0.008323434 = sum of:
0.008323434 = product of:
0.033293735 = sum of:
0.033293735 = weight(_text_:based in 2810) [ClassicSimilarity], result of:
0.033293735 = score(doc=2810,freq=4.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.23539014 = fieldWeight in 2810, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=2810)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Some Online Public Access Catalogues offer a ranking component. However, ranking there is merely text-based and is doomed to fail due to limited text in bibliographic data. The main assumption for the talk is that we are in a situation where the appropriate ranking factors for OPACs should be defined, while the implementation is no major problem. We must define what we want, and not so much focus on the technical work. Some deep thinking is necessary on the "perfect results set" and how we can achieve it through ranking. The talk presents a set of potential ranking factors and clustering possibilities for further discussion. A look at commercial Web search engines could provide us with ideas how ranking can be improved with additional factors. Search engines are way beyond pure text-based ranking and apply ranking factors in the groups like popularity, freshness, personalisation, etc. The talk describes the main factors used in search engines and how derivatives of these could be used for libraries' purposes. The goal of ranking is to provide the user with the best-suitable results on top of the results list. How can this goal be achieved with the library catalogue and also concerning the library's different collections and databases? The assumption is that ranking of such materials is a complex problem and is yet nowhere near solved. Libraries should focus on ranking to improve user experience.
-
Lewandowski, D.; Kerkmann, F.; Rümmele, S.; Sünkler, S.: ¬An empirical investigation on search engine ad disclosure (2018)
0.00
0.002059945 = product of:
0.00823978 = sum of:
0.00823978 = product of:
0.03295912 = sum of:
0.03295912 = weight(_text_:based in 4115) [ClassicSimilarity], result of:
0.03295912 = score(doc=4115,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.23302436 = fieldWeight in 4115, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0546875 = fieldNorm(doc=4115)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- This representative study of German search engine users (N?=?1,000) focuses on the ability of users to distinguish between organic results and advertisements on Google results pages. We combine questions about Google's business with task-based studies in which users were asked to distinguish between ads and organic results in screenshots of results pages. We find that only a small percentage of users can reliably distinguish between ads and organic results, and that user knowledge of Google's business model is very limited. We conclude that ads are insufficiently labelled as such, and that many users may click on ads assuming that they are selecting organic results.
-
Lewandowski, D.; Mayr, P.: Exploring the academic invisible Web (2006)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 3752) [ClassicSimilarity], result of:
0.023542227 = score(doc=3752,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 3752, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=3752)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose: To provide a critical review of Bergman's 2001 study on the Deep Web. In addition, we bring a new concept into the discussion, the Academic Invisible Web (AIW). We define the Academic Invisible Web as consisting of all databases and collections relevant to academia but not searchable by the general-purpose internet search engines. Indexing this part of the Invisible Web is central to scien-tific search engines. We provide an overview of approaches followed thus far. Design/methodology/approach: Discussion of measures and calculations, estima-tion based on informetric laws. Literature review on approaches for uncovering information from the Invisible Web. Findings: Bergman's size estimate of the Invisible Web is highly questionable. We demonstrate some major errors in the conceptual design of the Bergman paper. A new (raw) size estimate is given. Research limitations/implications: The precision of our estimate is limited due to a small sample size and lack of reliable data. Practical implications: We can show that no single library alone will be able to index the Academic Invisible Web. We suggest collaboration to accomplish this task. Originality/value: Provides library managers and those interested in developing academic search engines with data on the size and attributes of the Academic In-visible Web.
-
Lewandowski, D.: ¬The retrieval effectiveness of web search engines : considering results descriptions (2008)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 2345) [ClassicSimilarity], result of:
0.023542227 = score(doc=2345,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 2345, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=2345)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose - The purpose of this paper is to compare five major web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval effectiveness, taking into account not only the results, but also the results descriptions. Design/methodology/approach - The study uses real-life queries. Results are made anonymous and are randomized. Results are judged by the persons posing the original queries. Findings - The two major search engines, Google and Yahoo, perform best, and there are no significant differences between them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason for users perceiving this engine as superior. Research limitations/implications - The study is based on a user model where the user takes into account a certain amount of results rather systematically. This may not be the case in real life. Practical implications - The paper implies that search engines should focus on relevant descriptions. Searchers are advised to use other search engines in addition to Google. Originality/value - This is the first major study comparing results and descriptions systematically and proposes new retrieval measures to take into account results descriptions.
-
Lewandowski, D.; Mayr, P.: Exploring the academic invisible Web (2006)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 2580) [ClassicSimilarity], result of:
0.023542227 = score(doc=2580,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 2580, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=2580)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose: To provide a critical review of Bergman's 2001 study on the deep web. In addition, we bring a new concept into the discussion, the academic invisible web (AIW). We define the academic invisible web as consisting of all databases and collections relevant to academia but not searchable by the general-purpose internet search engines. Indexing this part of the invisible web is central to scientific search engines. We provide an overview of approaches followed thus far. Design/methodology/approach: Discussion of measures and calculations, estimation based on informetric laws. Literature review on approaches for uncovering information from the invisible web. Findings: Bergman's size estimate of the invisible web is highly questionable. We demonstrate some major errors in the conceptual design of the Bergman paper. A new (raw) size estimate is given. Research limitations/implications: The precision of our estimate is limited due to a small sample size and lack of reliable data. Practical implications: We can show that no single library alone will be able to index the academic invisible web. We suggest collaboration to accomplish this task. Originality/value: Provides library managers and those interested in developing academic search engines with data on the size and attributes of the academic invisible web.
-
Lewandowski, D.: ¬The retrieval effectiveness of search engines on navigational queries (2011)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 4537) [ClassicSimilarity], result of:
0.023542227 = score(doc=4537,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 4537, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=4537)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose - The purpose of this paper is to test major web search engines on their performance on navigational queries, i.e. searches for homepages. Design/methodology/approach - In total, 100 user queries are posed to six search engines (Google, Yahoo!, MSN, Ask, Seekport, and Exalead). Users described the desired pages, and the results position of these was recorded. Measured success and mean reciprocal rank are calculated. Findings - The performance of the major search engines Google, Yahoo!, and MSN was found to be the best, with around 90 per cent of queries answered correctly. Ask and Exalead performed worse but received good scores as well. Research limitations/implications - All queries were in German, and the German-language interfaces of the search engines were used. Therefore, the results are only valid for German queries. Practical implications - When designing a search engine to compete with the major search engines, care should be taken on the performance on navigational queries. Users can be influenced easily in their quality ratings of search engines based on this performance. Originality/value - This study systematically compares the major search engines on navigational queries and compares the findings with studies on the retrieval effectiveness of the engines on informational queries.
-
Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 3700) [ClassicSimilarity], result of:
0.023542227 = score(doc=3700,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 3700, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=3700)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose This paper demonstrates how to apply traditional information retrieval evaluation methods based on standards from the Text REtrieval Conference (TREC) and web search evaluation to all types of modern library information systems including online public access catalogs, discovery systems, and digital libraries that provide web search features to gather information from heterogeneous sources. Design/methodology/approach We apply conventional procedures from information retrieval evaluation to the library information system context considering the specific characteristics of modern library materials. Findings We introduce a framework consisting of five parts: (1) search queries, (2) search results, (3) assessors, (4) testing, and (5) data analysis. We show how to deal with comparability problems resulting from diverse document types, e.g., electronic articles vs. printed monographs and what issues need to be considered for retrieval tests in the library context. Practical implications The framework can be used as a guideline for conducting retrieval effectiveness studies in the library context. Originality/value Although a considerable amount of research has been done on information retrieval evaluation, and standards for conducting retrieval effectiveness studies do exist, to our knowledge this is the first attempt to provide a systematic framework for evaluating the retrieval effectiveness of twenty-first-century library information systems. We demonstrate which issues must be considered and what decisions must be made by researchers prior to a retrieval test.
