-
Lewandowski, D.: Evaluating the retrieval effectiveness of web search engines using a representative query sample (2015)
0.03
0.027245669 = product of:
0.054491337 = sum of:
0.0070626684 = product of:
0.028250674 = sum of:
0.028250674 = weight(_text_:based in 2157) [ClassicSimilarity], result of:
0.028250674 = score(doc=2157,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.19973516 = fieldWeight in 2157, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.046875 = fieldNorm(doc=2157)
0.25 = coord(1/4)
0.047428668 = product of:
0.094857335 = sum of:
0.094857335 = weight(_text_:assessment in 2157) [ClassicSimilarity], result of:
0.094857335 = score(doc=2157,freq=2.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.36599535 = fieldWeight in 2157, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.046875 = fieldNorm(doc=2157)
0.5 = coord(1/2)
0.5 = coord(2/4)
- Abstract
- Search engine retrieval effectiveness studies are usually small scale, using only limited query samples. Furthermore, queries are selected by the researchers. We address these issues by taking a random representative sample of 1,000 informational and 1,000 navigational queries from a major German search engine and comparing Google's and Bing's results based on this sample. Jurors were found through crowdsourcing, and data were collected using specialized software, the Relevance Assessment Tool (RAT). We found that although Google outperforms Bing in both query types, the difference in the performance for informational queries was rather low. However, for navigational queries, Google found the correct answer in 95.3% of cases, whereas Bing only found the correct answer 76.6% of the time. We conclude that search engine performance on navigational queries is of great importance, because users in this case can clearly identify queries that have returned correct results. So, performance on this query type may contribute to explaining user satisfaction with search engines.
-
Lewandowski, D.; Sünkler, S.: ¬Das Relevance Assessment Tool : eine modulare Software zur Unterstützung bei der Durchführung vielfältiger Studien mit Suchmaschinen (2019)
0.02
0.01676857 = product of:
0.06707428 = sum of:
0.06707428 = product of:
0.13414855 = sum of:
0.13414855 = weight(_text_:assessment in 5026) [ClassicSimilarity], result of:
0.13414855 = score(doc=5026,freq=4.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.51759565 = fieldWeight in 5026, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.046875 = fieldNorm(doc=5026)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- In diesem Artikel stellen wir eine Software vor, mit der sich Studien zu Such- und Informationssystemen realisieren lassen. Das Relevance Assessment Tool (RAT) soll umfangreiche Untersuchungen mit Daten von kommerziellen Suchmaschinen unterstützen. Die Software ist modular und webbasiert. Es lassen sich damit automatisiert Daten von Suchmaschinen erfassen. Dazu können Studien mit Fragen und Skalen flexibel gestaltet und die Informationsobjekte anhand der Fragen durch Juroren bewertet werden. Durch die Modularität lassen sich die einzelnen Komponenten für eine Vielzahl von Studien nutzen, die sich auf Web-Inhalte beziehen. So kann die Software auch für qualitative Inhaltsanalysen eingesetzt werden oder durch das automatisierte Scraping eine große Datenbasis an Web-Dokumenten liefern, die sich quantitativ in empirischen Studien analysieren lassen.
-
Schaer, P.; Mayr, P.; Sünkler, S.; Lewandowski, D.: How relevant is the long tail? : a relevance assessment study on million short (2016)
0.01
0.013973806 = product of:
0.055895224 = sum of:
0.055895224 = product of:
0.11179045 = sum of:
0.11179045 = weight(_text_:assessment in 3144) [ClassicSimilarity], result of:
0.11179045 = score(doc=3144,freq=4.0), product of:
0.25917634 = queryWeight, product of:
5.52102 = idf(docFreq=480, maxDocs=44218)
0.04694356 = queryNorm
0.43132967 = fieldWeight in 3144, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
5.52102 = idf(docFreq=480, maxDocs=44218)
0.0390625 = fieldNorm(doc=3144)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Abstract
- Users of web search engines are known to mostly focus on the top ranked results of the search engine result page. While many studies support this well known information seeking pattern only few studies concentrate on the question what users are missing by neglecting lower ranked results. To learn more about the relevance distributions in the so-called long tail we conducted a relevance assessment study with the Million Short long-tail web search engine. While we see a clear difference in the content between the head and the tail of the search engine result list we see no statistical significant differences in the binary relevance judgments and weak significant differences when using graded relevance. The tail contains different but still valuable results. We argue that the long tail can be a rich source for the diversification of web search engine result lists but it needs more evaluation to clearly describe the differences.
-
Lewandowski, D.; Sünkler, S.: What does Google recommend when you want to compare insurance offerings? (2019)
0.01
0.010893034 = product of:
0.021786068 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 5288) [ClassicSimilarity], result of:
0.023542227 = score(doc=5288,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 5288, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=5288)
0.25 = coord(1/4)
0.015900511 = product of:
0.031801023 = sum of:
0.031801023 = weight(_text_:22 in 5288) [ClassicSimilarity], result of:
0.031801023 = score(doc=5288,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.19345059 = fieldWeight in 5288, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=5288)
0.5 = coord(1/2)
0.5 = coord(2/4)
- Abstract
- Purpose The purpose of this paper is to describe a new method to improve the analysis of search engine results by considering the provider level as well as the domain level. This approach is tested by conducting a study using queries on the topic of insurance comparisons. Design/methodology/approach The authors conducted an empirical study that analyses the results of search queries aimed at comparing insurance companies. The authors used a self-developed software system that automatically queries commercial search engines and automatically extracts the content of the returned result pages for further data analysis. The data analysis was carried out using the KNIME Analytics Platform. Findings Google's top search results are served by only a few providers that frequently appear in these results. The authors show that some providers operate several domains on the same topic and that these domains appear for the same queries in the result lists. Research limitations/implications The authors demonstrate the feasibility of this approach and draw conclusions for further investigations from the empirical study. However, the study is a limited use case based on a limited number of search queries. Originality/value The proposed method allows large-scale analysis of the composition of the top results from commercial search engines. It allows using valid empirical data to determine what users actually see on the search engine result pages.
- Date
- 20. 1.2015 18:30:22
-
Lewandowski, D.: Query understanding (2011)
0.01
0.006360204 = product of:
0.025440816 = sum of:
0.025440816 = product of:
0.05088163 = sum of:
0.05088163 = weight(_text_:22 in 344) [ClassicSimilarity], result of:
0.05088163 = score(doc=344,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.30952093 = fieldWeight in 344, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0625 = fieldNorm(doc=344)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 18. 9.2018 18:22:18
-
Lewandowski, D.: ¬Die Macht der Suchmaschinen und ihr Einfluss auf unsere Entscheidungen (2014)
0.00
0.0047701527 = product of:
0.019080611 = sum of:
0.019080611 = product of:
0.038161222 = sum of:
0.038161222 = weight(_text_:22 in 1491) [ClassicSimilarity], result of:
0.038161222 = score(doc=1491,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.23214069 = fieldWeight in 1491, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=1491)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 22. 9.2014 18:54:11
-
Lewandowski, D.; Spree, U.: Ranking of Wikipedia articles in search engines revisited : fair ranking for reasonable quality? (2011)
0.00
0.003975128 = product of:
0.015900511 = sum of:
0.015900511 = product of:
0.031801023 = sum of:
0.031801023 = weight(_text_:22 in 444) [ClassicSimilarity], result of:
0.031801023 = score(doc=444,freq=2.0), product of:
0.16438834 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.04694356 = queryNorm
0.19345059 = fieldWeight in 444, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=444)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 30. 9.2012 19:27:22
-
Lewandowski, D.; Kerkmann, F.; Rümmele, S.; Sünkler, S.: ¬An empirical investigation on search engine ad disclosure (2018)
0.00
0.002059945 = product of:
0.00823978 = sum of:
0.00823978 = product of:
0.03295912 = sum of:
0.03295912 = weight(_text_:based in 4115) [ClassicSimilarity], result of:
0.03295912 = score(doc=4115,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.23302436 = fieldWeight in 4115, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0546875 = fieldNorm(doc=4115)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- This representative study of German search engine users (N?=?1,000) focuses on the ability of users to distinguish between organic results and advertisements on Google results pages. We combine questions about Google's business with task-based studies in which users were asked to distinguish between ads and organic results in screenshots of results pages. We find that only a small percentage of users can reliably distinguish between ads and organic results, and that user knowledge of Google's business model is very limited. We conclude that ads are insufficiently labelled as such, and that many users may click on ads assuming that they are selecting organic results.
-
Lewandowski, D.: ¬The retrieval effectiveness of search engines on navigational queries (2011)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 4537) [ClassicSimilarity], result of:
0.023542227 = score(doc=4537,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 4537, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=4537)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose - The purpose of this paper is to test major web search engines on their performance on navigational queries, i.e. searches for homepages. Design/methodology/approach - In total, 100 user queries are posed to six search engines (Google, Yahoo!, MSN, Ask, Seekport, and Exalead). Users described the desired pages, and the results position of these was recorded. Measured success and mean reciprocal rank are calculated. Findings - The performance of the major search engines Google, Yahoo!, and MSN was found to be the best, with around 90 per cent of queries answered correctly. Ask and Exalead performed worse but received good scores as well. Research limitations/implications - All queries were in German, and the German-language interfaces of the search engines were used. Therefore, the results are only valid for German queries. Practical implications - When designing a search engine to compete with the major search engines, care should be taken on the performance on navigational queries. Users can be influenced easily in their quality ratings of search engines based on this performance. Originality/value - This study systematically compares the major search engines on navigational queries and compares the findings with studies on the retrieval effectiveness of the engines on informational queries.
-
Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017)
0.00
0.0014713892 = product of:
0.005885557 = sum of:
0.005885557 = product of:
0.023542227 = sum of:
0.023542227 = weight(_text_:based in 3700) [ClassicSimilarity], result of:
0.023542227 = score(doc=3700,freq=2.0), product of:
0.14144066 = queryWeight, product of:
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.04694356 = queryNorm
0.16644597 = fieldWeight in 3700, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.0129938 = idf(docFreq=5906, maxDocs=44218)
0.0390625 = fieldNorm(doc=3700)
0.25 = coord(1/4)
0.25 = coord(1/4)
- Abstract
- Purpose This paper demonstrates how to apply traditional information retrieval evaluation methods based on standards from the Text REtrieval Conference (TREC) and web search evaluation to all types of modern library information systems including online public access catalogs, discovery systems, and digital libraries that provide web search features to gather information from heterogeneous sources. Design/methodology/approach We apply conventional procedures from information retrieval evaluation to the library information system context considering the specific characteristics of modern library materials. Findings We introduce a framework consisting of five parts: (1) search queries, (2) search results, (3) assessors, (4) testing, and (5) data analysis. We show how to deal with comparability problems resulting from diverse document types, e.g., electronic articles vs. printed monographs and what issues need to be considered for retrieval tests in the library context. Practical implications The framework can be used as a guideline for conducting retrieval effectiveness studies in the library context. Originality/value Although a considerable amount of research has been done on information retrieval evaluation, and standards for conducting retrieval effectiveness studies do exist, to our knowledge this is the first attempt to provide a systematic framework for evaluating the retrieval effectiveness of twenty-first-century library information systems. We demonstrate which issues must be considered and what decisions must be made by researchers prior to a retrieval test.
