Document (#32836)

Author

Ruthven, I.
Baillie, M.
Elsweiler, D.

Title

¬The relative effects of knowledge, interest and confidence in assessing relevance

Source

Journal of documentation. 63(2007) no.4, S.482-504

Year

2007

Abstract

Purpose - The purpose of this paper is to examine how different aspects of an assessor's context, in particular their knowledge of a search topic, their interest in the search topic and their confidence in assessing relevance for a topic, affect the relevance judgements made and the assessor's ability to predict which documents they will assess as being relevant. Design/methodology/approach - The study was conducted as part of the Text REtrieval Conference (TREC) HARD track. Using a specially constructed questionnaire information was sought on TREC assessors' personal context and, using the TREC assessments gathered, the responses were correlated to the questionnaire questions and the final relevance decisions. Findings - This study found that each of the three factors (interest, knowledge and confidence) had an affect on how many documents were assessed as relevant and the balance between how many documents were marked as marginally or highly relevant. Also these factors are shown to affect an assessors' ability to predict what information they will finally mark as being relevant. Research limitations/implications - The major limitation is that the research is conducted within the TREC initiative. This means that we can report on results but cannot report on discussions with the assessors. The research implications are numerous but mainly on the effect of personal context on the outcomes of a user study. Practical implications - One major consequence is that we should take more account of how we construct search tasks for IIR evaluation to create tasks that are interesting and relevant to experimental subjects. Originality/value - Examining different search variables within one study to compare the relative effects on these variables on the search outcomes.

Theme

Retrievalstudien

Similar documents (author)

1. Elsweiler, D.; Ruthven, I.; Jones, C.: Towards memory supporting personal information management tools (2007) 2.47

   2.465322 = sum of:
     2.465322 = product of:
       3.697983 = sum of:
         1.3889117 = weight(author_txt:ruthven in 5057) [ClassicSimilarity], result of:
           1.3889117 = score(doc=5057,freq=1.0), product of:
             0.4499692 = queryWeight, product of:
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.054666616 = queryNorm
             3.0866818 = fieldWeight in 5057, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.375 = fieldNorm(doc=5057)
         2.3090713 = weight(author_txt:elsweiler in 5057) [ClassicSimilarity], result of:
           2.3090713 = score(doc=5057,freq=1.0), product of:
             0.63147753 = queryWeight, product of:
               1.1846431 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.054666616 = queryNorm
             3.6566167 = fieldWeight in 5057, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.375 = fieldNorm(doc=5057)
       0.6666667 = coord(2/3)
   

2. Baillie, M.; Azzopardi, L.; Ruthven, I.: Evaluating epistemic uncertainty under incomplete assessments (2008) 2.47

   2.465322 = sum of:
     2.465322 = product of:
       3.697983 = sum of:
         1.3889117 = weight(author_txt:ruthven in 2065) [ClassicSimilarity], result of:
           1.3889117 = score(doc=2065,freq=1.0), product of:
             0.4499692 = queryWeight, product of:
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.054666616 = queryNorm
             3.0866818 = fieldWeight in 2065, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.375 = fieldNorm(doc=2065)
         2.3090713 = weight(author_txt:baillie in 2065) [ClassicSimilarity], result of:
           2.3090713 = score(doc=2065,freq=1.0), product of:
             0.63147753 = queryWeight, product of:
               1.1846431 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.054666616 = queryNorm
             3.6566167 = fieldWeight in 2065, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.375 = fieldNorm(doc=2065)
       0.6666667 = coord(2/3)
   

3. Ruthven, I.; Baillie, M.; Azzopardi, L.; Bierig, R.; Nicol, E.; Sweeney, S.; Yaciki, M.: Contextual factors affecting the utility of surrogates within exploratory search (2008) 1.64

   1.643548 = sum of:
     1.643548 = product of:
       2.465322 = sum of:
         0.92594117 = weight(author_txt:ruthven in 2042) [ClassicSimilarity], result of:
           0.92594117 = score(doc=2042,freq=1.0), product of:
             0.4499692 = queryWeight, product of:
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.054666616 = queryNorm
             2.057788 = fieldWeight in 2042, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.25 = fieldNorm(doc=2042)
         1.5393808 = weight(author_txt:baillie in 2042) [ClassicSimilarity], result of:
           1.5393808 = score(doc=2042,freq=1.0), product of:
             0.63147753 = queryWeight, product of:
               1.1846431 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.054666616 = queryNorm
             2.4377444 = fieldWeight in 2042, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.25 = fieldNorm(doc=2042)
       0.6666667 = coord(2/3)
   

4. Elsweiler, D.; Harvey, M.: Engaging and maintaining a sense of being informed : understanding the tasks motivating twitter search (2015) 1.03

   1.0262539 = sum of:
     1.0262539 = product of:
       3.0787616 = sum of:
         3.0787616 = weight(author_txt:elsweiler in 1635) [ClassicSimilarity], result of:
           3.0787616 = score(doc=1635,freq=1.0), product of:
             0.63147753 = queryWeight, product of:
               1.1846431 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.054666616 = queryNorm
             4.8754888 = fieldWeight in 1635, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.5 = fieldNorm(doc=1635)
       0.33333334 = coord(1/3)
   

5. Elsweiler, D.; Hammwöhner, R.: Information Behavior : ein zentrales Forschungsthema der Informationswissenschaft (2015) 1.03

   1.0262539 = sum of:
     1.0262539 = product of:
       3.0787616 = sum of:
         3.0787616 = weight(author_txt:elsweiler in 1713) [ClassicSimilarity], result of:
           3.0787616 = score(doc=1713,freq=1.0), product of:
             0.63147753 = queryWeight, product of:
               1.1846431 = boost
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.054666616 = queryNorm
             4.8754888 = fieldWeight in 1713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.7509775 = idf(docFreq=6, maxDocs=44218)
               0.5 = fieldNorm(doc=1713)
       0.33333334 = coord(1/3)
   

Similar documents (content)

1. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.50

   0.49712807 = sum of:
     0.49712807 = product of:
       1.2428201 = sum of:
         0.019443048 = weight(abstract_txt:were in 4540) [ClassicSimilarity], result of:
           0.019443048 = score(doc=4540,freq=1.0), product of:
             0.08476386 = queryWeight, product of:
               1.0330093 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.022357997 = queryNorm
             0.22937898 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.015104604 = weight(abstract_txt:that in 4540) [ClassicSimilarity], result of:
           0.015104604 = score(doc=4540,freq=3.0), product of:
             0.05888661 = queryWeight, product of:
               1.1115564 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.022357997 = queryNorm
             0.2565032 = fieldWeight in 4540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.063757755 = weight(abstract_txt:questionnaire in 4540) [ClassicSimilarity], result of:
           0.063757755 = score(doc=4540,freq=1.0), product of:
             0.16344039 = queryWeight, product of:
               1.171205 = boost
               6.241566 = idf(docFreq=233, maxDocs=44218)
               0.022357997 = queryNorm
             0.3900979 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.241566 = idf(docFreq=233, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.050069295 = weight(abstract_txt:implications in 4540) [ClassicSimilarity], result of:
           0.050069295 = score(doc=4540,freq=2.0), product of:
             0.1263976 = queryWeight, product of:
               1.2614448 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.022357997 = queryNorm
             0.39612538 = fieldWeight in 4540, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.04209589 = weight(abstract_txt:study in 4540) [ClassicSimilarity], result of:
           0.04209589 = score(doc=4540,freq=4.0), product of:
             0.09836046 = queryWeight, product of:
               1.2849271 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.022357997 = queryNorm
             0.42797574 = fieldWeight in 4540, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.08170009 = weight(abstract_txt:affect in 4540) [ClassicSimilarity], result of:
           0.08170009 = score(doc=4540,freq=1.0), product of:
             0.22072397 = queryWeight, product of:
               1.6669539 = boost
               5.9223356 = idf(docFreq=321, maxDocs=44218)
               0.022357997 = queryNorm
             0.37014598 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9223356 = idf(docFreq=321, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.032087956 = weight(abstract_txt:search in 4540) [ClassicSimilarity], result of:
           0.032087956 = score(doc=4540,freq=1.0), product of:
             0.1403499 = queryWeight, product of:
               1.7160474 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.022357997 = queryNorm
             0.22862828 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.20864487 = weight(abstract_txt:relevance in 4540) [ClassicSimilarity], result of:
           0.20864487 = score(doc=4540,freq=11.0), product of:
             0.20408995 = queryWeight, product of:
               1.8508834 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.022357997 = queryNorm
             1.0223181 = fieldWeight in 4540, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.45676488 = weight(abstract_txt:assessors in 4540) [ClassicSimilarity], result of:
           0.45676488 = score(doc=4540,freq=3.0), product of:
             0.4820838 = queryWeight, product of:
               2.463543 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.022357997 = queryNorm
             0.94748026 = fieldWeight in 4540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.27315181 = weight(abstract_txt:trec in 4540) [ClassicSimilarity], result of:
           0.27315181 = score(doc=4540,freq=3.0), product of:
             0.3766255 = queryWeight, product of:
               2.5143347 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.022357997 = queryNorm
             0.7252611 = fieldWeight in 4540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
       0.4 = coord(10/25)
   

2. Ruthven, I.: Relevance behaviour in TREC (2014) 0.39

   0.38527873 = sum of:
     0.38527873 = product of:
       1.2039961 = sum of:
         0.023072662 = weight(abstract_txt:that in 1785) [ClassicSimilarity], result of:
           0.023072662 = score(doc=1785,freq=7.0), product of:
             0.05888661 = queryWeight, product of:
               1.1115564 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.022357997 = queryNorm
             0.3918151 = fieldWeight in 1785, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.03540434 = weight(abstract_txt:implications in 1785) [ClassicSimilarity], result of:
           0.03540434 = score(doc=1785,freq=1.0), product of:
             0.1263976 = queryWeight, product of:
               1.2614448 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.022357997 = queryNorm
             0.28010294 = fieldWeight in 1785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.05241869 = weight(abstract_txt:interest in 1785) [ClassicSimilarity], result of:
           0.05241869 = score(doc=1785,freq=1.0), product of:
             0.1641945 = queryWeight, product of:
               1.4377327 = boost
               5.1079607 = idf(docFreq=726, maxDocs=44218)
               0.022357997 = queryNorm
             0.31924754 = fieldWeight in 1785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1079607 = idf(docFreq=726, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.115541376 = weight(abstract_txt:affect in 1785) [ClassicSimilarity], result of:
           0.115541376 = score(doc=1785,freq=2.0), product of:
             0.22072397 = queryWeight, product of:
               1.6669539 = boost
               5.9223356 = idf(docFreq=321, maxDocs=44218)
               0.022357997 = queryNorm
             0.52346545 = fieldWeight in 1785, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9223356 = idf(docFreq=321, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.032087956 = weight(abstract_txt:search in 1785) [ClassicSimilarity], result of:
           0.032087956 = score(doc=1785,freq=1.0), product of:
             0.1403499 = queryWeight, product of:
               1.7160474 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.022357997 = queryNorm
             0.22862828 = fieldWeight in 1785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.20864487 = weight(abstract_txt:relevance in 1785) [ClassicSimilarity], result of:
           0.20864487 = score(doc=1785,freq=11.0), product of:
             0.20408995 = queryWeight, product of:
               1.8508834 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.022357997 = queryNorm
             1.0223181 = fieldWeight in 1785, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.26371333 = weight(abstract_txt:assessors in 1785) [ClassicSimilarity], result of:
           0.26371333 = score(doc=1785,freq=1.0), product of:
             0.4820838 = queryWeight, product of:
               2.463543 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.022357997 = queryNorm
             0.547028 = fieldWeight in 1785, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
         0.47311285 = weight(abstract_txt:trec in 1785) [ClassicSimilarity], result of:
           0.47311285 = score(doc=1785,freq=9.0), product of:
             0.3766255 = queryWeight, product of:
               2.5143347 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.022357997 = queryNorm
             1.2561891 = fieldWeight in 1785, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=1785)
       0.32 = coord(8/25)
   

3. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.28

   0.28493947 = sum of:
     0.28493947 = product of:
       1.017641 = sum of:
         0.015104604 = weight(abstract_txt:that in 2591) [ClassicSimilarity], result of:
           0.015104604 = score(doc=2591,freq=3.0), product of:
             0.05888661 = queryWeight, product of:
               1.1115564 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.022357997 = queryNorm
             0.2565032 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.04768784 = weight(abstract_txt:documents in 2591) [ClassicSimilarity], result of:
           0.04768784 = score(doc=2591,freq=3.0), product of:
             0.106888875 = queryWeight, product of:
               1.160019 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.022357997 = queryNorm
             0.44614407 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.036456108 = weight(abstract_txt:study in 2591) [ClassicSimilarity], result of:
           0.036456108 = score(doc=2591,freq=3.0), product of:
             0.09836046 = queryWeight, product of:
               1.2849271 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.022357997 = queryNorm
             0.37063786 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.07215896 = weight(abstract_txt:topic in 2591) [ClassicSimilarity], result of:
           0.07215896 = score(doc=2591,freq=2.0), product of:
             0.16126917 = queryWeight, product of:
               1.4248677 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.022357997 = queryNorm
             0.44744426 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.16644101 = weight(abstract_txt:relevance in 2591) [ClassicSimilarity], result of:
           0.16644101 = score(doc=2591,freq=7.0), product of:
             0.20408995 = queryWeight, product of:
               1.8508834 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.022357997 = queryNorm
             0.8155277 = fieldWeight in 2591, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.45676488 = weight(abstract_txt:assessors in 2591) [ClassicSimilarity], result of:
           0.45676488 = score(doc=2591,freq=3.0), product of:
             0.4820838 = queryWeight, product of:
               2.463543 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.022357997 = queryNorm
             0.94748026 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.22302754 = weight(abstract_txt:trec in 2591) [ClassicSimilarity], result of:
           0.22302754 = score(doc=2591,freq=2.0), product of:
             0.3766255 = queryWeight, product of:
               2.5143347 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.022357997 = queryNorm
             0.5921732 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
       0.28 = coord(7/25)
   

4. Osman, D.J.; Yearwood, J.; Vamplew, P.: Automated opinion detection : implications of the level of agreement between human raters (2010) 0.26

   0.26168227 = sum of:
     0.26168227 = product of:
       1.0903429 = sum of:
         0.045473497 = weight(abstract_txt:ability in 4232) [ClassicSimilarity], result of:
           0.045473497 = score(doc=4232,freq=1.0), product of:
             0.1304697 = queryWeight, product of:
               1.0464249 = boost
               5.5765896 = idf(docFreq=454, maxDocs=44218)
               0.022357997 = queryNorm
             0.34853685 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5765896 = idf(docFreq=454, maxDocs=44218)
               0.0625 = fieldNorm(doc=4232)
         0.012332858 = weight(abstract_txt:that in 4232) [ClassicSimilarity], result of:
           0.012332858 = score(doc=4232,freq=2.0), product of:
             0.05888661 = queryWeight, product of:
               1.1115564 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.022357997 = queryNorm
             0.20943399 = fieldWeight in 4232, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=4232)
         0.07791426 = weight(abstract_txt:assessing in 4232) [ClassicSimilarity], result of:
           0.07791426 = score(doc=4232,freq=1.0), product of:
             0.18681656 = queryWeight, product of:
               1.2521632 = boost
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.022357997 = queryNorm
             0.4170629 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.0625 = fieldNorm(doc=4232)
         0.051024094 = weight(abstract_txt:topic in 4232) [ClassicSimilarity], result of:
           0.051024094 = score(doc=4232,freq=1.0), product of:
             0.16126917 = queryWeight, product of:
               1.4248677 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.022357997 = queryNorm
             0.31639087 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.0625 = fieldNorm(doc=4232)
         0.74589396 = weight(abstract_txt:assessors in 4232) [ClassicSimilarity], result of:
           0.74589396 = score(doc=4232,freq=8.0), product of:
             0.4820838 = queryWeight, product of:
               2.463543 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.022357997 = queryNorm
             1.5472288 = fieldWeight in 4232, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=4232)
         0.15770428 = weight(abstract_txt:trec in 4232) [ClassicSimilarity], result of:
           0.15770428 = score(doc=4232,freq=1.0), product of:
             0.3766255 = queryWeight, product of:
               2.5143347 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.022357997 = queryNorm
             0.4187297 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=4232)
       0.24 = coord(6/25)
   

5. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 0.25

   0.2514318 = sum of:
     0.2514318 = product of:
       0.7857244 = sum of:
         0.018880755 = weight(abstract_txt:that in 3145) [ClassicSimilarity], result of:
           0.018880755 = score(doc=3145,freq=3.0), product of:
             0.05888661 = queryWeight, product of:
               1.1115564 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.022357997 = queryNorm
             0.320629 = fieldWeight in 3145, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.034415737 = weight(abstract_txt:documents in 3145) [ClassicSimilarity], result of:
           0.034415737 = score(doc=3145,freq=1.0), product of:
             0.106888875 = queryWeight, product of:
               1.160019 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.022357997 = queryNorm
             0.32197678 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.04018958 = weight(abstract_txt:context in 3145) [ClassicSimilarity], result of:
           0.04018958 = score(doc=3145,freq=1.0), product of:
             0.11853232 = queryWeight, product of:
               1.2215669 = boost
               4.339969 = idf(docFreq=1566, maxDocs=44218)
               0.022357997 = queryNorm
             0.3390601 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.339969 = idf(docFreq=1566, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.09739282 = weight(abstract_txt:assessing in 3145) [ClassicSimilarity], result of:
           0.09739282 = score(doc=3145,freq=1.0), product of:
             0.18681656 = queryWeight, product of:
               1.2521632 = boost
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.022357997 = queryNorm
             0.5213286 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.026309932 = weight(abstract_txt:study in 3145) [ClassicSimilarity], result of:
           0.026309932 = score(doc=3145,freq=1.0), product of:
             0.09836046 = queryWeight, product of:
               1.2849271 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.022357997 = queryNorm
             0.26748484 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.15727198 = weight(abstract_txt:relevance in 3145) [ClassicSimilarity], result of:
           0.15727198 = score(doc=3145,freq=4.0), product of:
             0.20408995 = queryWeight, product of:
               1.8508834 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.022357997 = queryNorm
             0.7706013 = fieldWeight in 3145, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.081621945 = weight(abstract_txt:relevant in 3145) [ClassicSimilarity], result of:
           0.081621945 = score(doc=3145,freq=1.0), product of:
             0.22537999 = queryWeight, product of:
               2.1746078 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.022357997 = queryNorm
             0.36215258 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.32964167 = weight(abstract_txt:assessors in 3145) [ClassicSimilarity], result of:
           0.32964167 = score(doc=3145,freq=1.0), product of:
             0.4820838 = queryWeight, product of:
               2.463543 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.022357997 = queryNorm
             0.683785 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
       0.32 = coord(8/25)