-
Ruthven, I.: Relevance behaviour in TREC (2014)
0.01
0.014458986 = product of:
0.057835944 = sum of:
0.057835944 = weight(_text_:data in 1785) [ClassicSimilarity], result of:
0.057835944 = score(doc=1785,freq=10.0), product of:
0.14807065 = queryWeight, product of:
3.1620505 = idf(docFreq=5088, maxDocs=44218)
0.046827413 = queryNorm
0.39059696 = fieldWeight in 1785, product of:
3.1622777 = tf(freq=10.0), with freq of:
10.0 = termFreq=10.0
3.1620505 = idf(docFreq=5088, maxDocs=44218)
0.0390625 = fieldNorm(doc=1785)
0.25 = coord(1/4)
- Abstract
- Purpose - The purpose of this paper is to examine how various types of TREC data can be used to better understand relevance and serve as test-bed for exploring relevance. The author proposes that there are many interesting studies that can be performed on the TREC data collections that are not directly related to evaluating systems but to learning more about human judgements of information and relevance and that these studies can provide useful research questions for other types of investigation. Design/methodology/approach - Through several case studies the author shows how existing data from TREC can be used to learn more about the factors that may affect relevance judgements and interactive search decisions and answer new research questions for exploring relevance. Findings - The paper uncovers factors, such as familiarity, interest and strictness of relevance criteria, that affect the nature of relevance assessments within TREC, contrasting these against findings from user studies of relevance. Research limitations/implications - The research only considers certain uses of TREC data and assessment given by professional relevance assessors but motivates further exploration of the TREC data so that the research community can further exploit the effort involved in the construction of TREC test collections. Originality/value - The paper presents an original viewpoint on relevance investigations and TREC itself by motivating TREC as a source of inspiration on understanding relevance rather than purely as a source of evaluation material.
-
Balatsoukas, P.; Ruthven, I.: ¬An eye-tracking approach to the analysis of relevance judgments on the Web : the case of Google search engine (2012)
0.01
0.009144665 = product of:
0.03657866 = sum of:
0.03657866 = weight(_text_:data in 379) [ClassicSimilarity], result of:
0.03657866 = score(doc=379,freq=4.0), product of:
0.14807065 = queryWeight, product of:
3.1620505 = idf(docFreq=5088, maxDocs=44218)
0.046827413 = queryNorm
0.24703519 = fieldWeight in 379, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.1620505 = idf(docFreq=5088, maxDocs=44218)
0.0390625 = fieldNorm(doc=379)
0.25 = coord(1/4)
- Abstract
- Eye movement data can provide an in-depth view of human reasoning and the decision-making process, and modern information retrieval (IR) research can benefit from the analysis of this type of data. The aim of this research was to examine the relationship between relevance criteria use and visual behavior in the context of predictive relevance judgments. To address this objective, a multimethod research design was employed that involved observation of participants' eye movements, talk-aloud protocols, and postsearch interviews. Specifically, the results reported in this article came from the analysis of 281 predictive relevance judgments made by 24 participants using the Google search engine. We present a novel stepwise methodological framework for the analysis of relevance judgments and eye movements on the Web and show new patterns of relevance criteria use during predictive relevance judgment. For example, the findings showed an effect of ranking order and surrogate components (Title, Summary, and URL) on the use of relevance criteria. Also, differences were observed in the cognitive effort spent between very relevant and not relevant judgments. We conclude with the implications of this study for IR research.
-
Ruthven, I.; Baillie, M.; Azzopardi, L.; Bierig, R.; Nicol, E.; Sweeney, S.; Yaciki, M.: Contextual factors affecting the utility of surrogates within exploratory search (2008)
0.01
0.007418666 = product of:
0.029674664 = sum of:
0.029674664 = product of:
0.05934933 = sum of:
0.05934933 = weight(_text_:processing in 2042) [ClassicSimilarity], result of:
0.05934933 = score(doc=2042,freq=2.0), product of:
0.18956426 = queryWeight, product of:
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046827413 = queryNorm
0.3130829 = fieldWeight in 2042, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.0546875 = fieldNorm(doc=2042)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Source
- Information processing and management. 44(2008) no.2, S.437-462
-
White, R.W.; Jose, J.M.; Ruthven, I.: ¬An implicit feedback approach for interactive information retrieval (2006)
0.01
0.0063588563 = product of:
0.025435425 = sum of:
0.025435425 = product of:
0.05087085 = sum of:
0.05087085 = weight(_text_:processing in 964) [ClassicSimilarity], result of:
0.05087085 = score(doc=964,freq=2.0), product of:
0.18956426 = queryWeight, product of:
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046827413 = queryNorm
0.26835677 = fieldWeight in 964, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046875 = fieldNorm(doc=964)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Source
- Information processing and management. 42(2006) no.1, S.166-190
-
Baillie, M.; Azzopardi, L.; Ruthven, I.: Evaluating epistemic uncertainty under incomplete assessments (2008)
0.01
0.0063588563 = product of:
0.025435425 = sum of:
0.025435425 = product of:
0.05087085 = sum of:
0.05087085 = weight(_text_:processing in 2065) [ClassicSimilarity], result of:
0.05087085 = score(doc=2065,freq=2.0), product of:
0.18956426 = queryWeight, product of:
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046827413 = queryNorm
0.26835677 = fieldWeight in 2065, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046875 = fieldNorm(doc=2065)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Source
- Information processing and management. 44(2008) no.2, S.811-837
-
White, R.W.; Jose, J.M.; Ruthven, I.: ¬A task-oriented study on the influencing effects of query-biased summarisation in web searching (2003)
0.01
0.005299047 = product of:
0.021196188 = sum of:
0.021196188 = product of:
0.042392377 = sum of:
0.042392377 = weight(_text_:processing in 1081) [ClassicSimilarity], result of:
0.042392377 = score(doc=1081,freq=2.0), product of:
0.18956426 = queryWeight, product of:
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046827413 = queryNorm
0.22363065 = fieldWeight in 1081, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.0390625 = fieldNorm(doc=1081)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Source
- Information processing and management. 39(2003) no.5, S.689-706
-
Ruthven, I.: ¬The language of information need : differentiating conscious and formalized information needs (2019)
0.01
0.005299047 = product of:
0.021196188 = sum of:
0.021196188 = product of:
0.042392377 = sum of:
0.042392377 = weight(_text_:processing in 5035) [ClassicSimilarity], result of:
0.042392377 = score(doc=5035,freq=2.0), product of:
0.18956426 = queryWeight, product of:
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046827413 = queryNorm
0.22363065 = fieldWeight in 5035, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.0390625 = fieldNorm(doc=5035)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Source
- Information processing and management. 56(2019) no.1, S.77-90
-
Borlund, P.; Ruthven, I.: Introduction to the special issue on evaluating interactive information retrieval systems (2008)
0.00
0.004239238 = product of:
0.016956951 = sum of:
0.016956951 = product of:
0.033913903 = sum of:
0.033913903 = weight(_text_:processing in 2019) [ClassicSimilarity], result of:
0.033913903 = score(doc=2019,freq=2.0), product of:
0.18956426 = queryWeight, product of:
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.046827413 = queryNorm
0.17890452 = fieldWeight in 2019, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
4.048147 = idf(docFreq=2097, maxDocs=44218)
0.03125 = fieldNorm(doc=2019)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Source
- Information processing and management. 44(2008) no.1, S.1-3
-
Belabbes, M.A.; Ruthven, I.; Moshfeghi, Y.; Rasmussen Pennington, D.: Information overload : a concept analysis (2023)
0.00
0.0039652926 = product of:
0.01586117 = sum of:
0.01586117 = product of:
0.03172234 = sum of:
0.03172234 = weight(_text_:22 in 950) [ClassicSimilarity], result of:
0.03172234 = score(doc=950,freq=2.0), product of:
0.16398162 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046827413 = queryNorm
0.19345059 = fieldWeight in 950, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=950)
0.5 = coord(1/2)
0.25 = coord(1/4)
- Date
- 22. 4.2023 19:27:56
