Document (#12074)

Author

Harter, S.P.

Title

Variations in relevance assessments and the measurement of retrieval effectiveness

Source

Journal of the American Society for Information Science. 47(1996) no.1, S.37-49

Year

1996

Abstract

The purpose of this article is to bring attention to the problem of variations in relevance assessments and the effects that these may have on measures of retrieval effectiveness. Through an analytical review of the literature, I show that despite known wide variations in relevance assessments in experimental test collections, their effects on the measurement of retrieval performance are almost completely unstudied. I will further argue that what we know about tha many variables that have been found to affect relevance assessments under experimental conditions, as well as our new understanding of psychological, situational, user-based relevance, point to a single conclusion. We can no longer rest the evaluation of information retrieval systems on the assumption that such variations do not significantly affect the measurement of information retrieval performance. A series of thourough, rigorous, and extensive tests is needed, of precisely how, and under what conditions, variations in relevance assessments do, and do not, affect measures of retrieval performance. We need to develop approaches to evaluation that are sensitive to these variations and to human factors and individual differences more generally. Our approaches to evaluation must reflect the real world of real users

Theme

Retrievalstudien

Similar documents (author)

1. Harter, S.P.: Search term combinations and retrieval overlap : a proposed methodology and case study (1990) 5.51

   5.5070662 = sum of:
     5.5070662 = weight(author_txt:harter in 339) [ClassicSimilarity], result of:
       5.5070662 = fieldWeight in 339, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.811306 = idf(docFreq=16, maxDocs=41962)
         0.625 = fieldNorm(doc=339)
   

2. Harter, S.P.: Detrimental effects of searching with precoordinated terms (1988) 5.51

   5.5070662 = sum of:
     5.5070662 = weight(author_txt:harter in 366) [ClassicSimilarity], result of:
       5.5070662 = fieldWeight in 366, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.811306 = idf(docFreq=16, maxDocs=41962)
         0.625 = fieldNorm(doc=366)
   

3. Harter, S.P.: Psychological relevance and information science (1992) 5.51

   5.5070662 = sum of:
     5.5070662 = weight(author_txt:harter in 5355) [ClassicSimilarity], result of:
       5.5070662 = fieldWeight in 5355, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.811306 = idf(docFreq=16, maxDocs=41962)
         0.625 = fieldNorm(doc=5355)
   

4. Harter, S.P.: ¬The Cranfield II relevance assessments : a critical evaluation (1971) 5.51

   5.5070662 = sum of:
     5.5070662 = weight(author_txt:harter in 5364) [ClassicSimilarity], result of:
       5.5070662 = fieldWeight in 5364, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.811306 = idf(docFreq=16, maxDocs=41962)
         0.625 = fieldNorm(doc=5364)
   

5. Harter, S.P.: ¬A probabilistic approach to keyword indexing (1975) 5.51

   5.5070662 = sum of:
     5.5070662 = weight(author_txt:harter in 5481) [ClassicSimilarity], result of:
       5.5070662 = fieldWeight in 5481, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.811306 = idf(docFreq=16, maxDocs=41962)
         0.625 = fieldNorm(doc=5481)
   

Similar documents (content)

1. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 0.54

   0.5407671 = sum of:
     0.5407671 = product of:
       1.5021307 = sum of:
         0.048993483 = weight(abstract_txt:effectiveness in 3308) [ClassicSimilarity], result of:
           0.048993483 = score(doc=3308,freq=1.0), product of:
             0.101904035 = queryWeight, product of:
               1.4310578 = boost
               5.1283264 = idf(docFreq=675, maxDocs=41962)
               0.013885405 = queryNorm
             0.4807806 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1283264 = idf(docFreq=675, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.050901376 = weight(abstract_txt:under in 3308) [ClassicSimilarity], result of:
           0.050901376 = score(doc=3308,freq=1.0), product of:
             0.104532704 = queryWeight, product of:
               1.4493977 = boost
               5.1940494 = idf(docFreq=632, maxDocs=41962)
               0.013885405 = queryNorm
             0.4869421 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1940494 = idf(docFreq=632, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.091342136 = weight(abstract_txt:conditions in 3308) [ClassicSimilarity], result of:
           0.091342136 = score(doc=3308,freq=1.0), product of:
             0.15436421 = queryWeight, product of:
               1.7613059 = boost
               6.311801 = idf(docFreq=206, maxDocs=41962)
               0.013885405 = queryNorm
             0.5917313 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.311801 = idf(docFreq=206, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.049629997 = weight(abstract_txt:evaluation in 3308) [ClassicSimilarity], result of:
           0.049629997 = score(doc=3308,freq=1.0), product of:
             0.11765916 = queryWeight, product of:
               1.8833021 = boost
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.013885405 = queryNorm
             0.42181158 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.09554874 = weight(abstract_txt:performance in 3308) [ClassicSimilarity], result of:
           0.09554874 = score(doc=3308,freq=3.0), product of:
             0.12625226 = queryWeight, product of:
               1.9508626 = boost
               4.66073 = idf(docFreq=1078, maxDocs=41962)
               0.013885405 = queryNorm
             0.75680816 = fieldWeight in 3308, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.66073 = idf(docFreq=1078, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.16755523 = weight(abstract_txt:affect in 3308) [ClassicSimilarity], result of:
           0.16755523 = score(doc=3308,freq=2.0), product of:
             0.21016395 = queryWeight, product of:
               2.5170174 = boost
               6.013308 = idf(docFreq=278, maxDocs=41962)
               0.013885405 = queryNorm
             0.79725957 = fieldWeight in 3308, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.013308 = idf(docFreq=278, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.090389624 = weight(abstract_txt:retrieval in 3308) [ClassicSimilarity], result of:
           0.090389624 = score(doc=3308,freq=4.0), product of:
             0.13927248 = queryWeight, product of:
               2.897709 = boost
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.013885405 = queryNorm
             0.6490128 = fieldWeight in 3308, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.32423863 = weight(abstract_txt:relevance in 3308) [ClassicSimilarity], result of:
           0.32423863 = score(doc=3308,freq=6.0), product of:
             0.28510118 = queryWeight, product of:
               4.1459265 = boost
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.013885405 = queryNorm
             1.1372757 = fieldWeight in 3308, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
         0.58353144 = weight(abstract_txt:variations in 3308) [ClassicSimilarity], result of:
           0.58353144 = score(doc=3308,freq=3.0), product of:
             0.5314658 = queryWeight, product of:
               5.660565 = boost
               6.761718 = idf(docFreq=131, maxDocs=41962)
               0.013885405 = queryNorm
             1.0979661 = fieldWeight in 3308, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.761718 = idf(docFreq=131, maxDocs=41962)
               0.09375 = fieldNorm(doc=3308)
       0.36 = coord(9/25)
   

2. Golub, K.; Soergel, D.; Buchanan, G.; Tudhope, D.; Lykke, M.; Hiom, D.: ¬A framework for evaluating automatic indexing or classification in the context of retrieval (2016) 0.26

   0.25752047 = sum of:
     0.25752047 = product of:
       0.64380115 = sum of:
         0.0354363 = weight(abstract_txt:approaches in 312) [ClassicSimilarity], result of:
           0.0354363 = score(doc=312,freq=2.0), product of:
             0.08539824 = queryWeight, product of:
               1.3100442 = boost
               4.694663 = idf(docFreq=1042, maxDocs=41962)
               0.013885405 = queryNorm
             0.4149535 = fieldWeight in 312, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.694663 = idf(docFreq=1042, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.037261043 = weight(abstract_txt:real in 312) [ClassicSimilarity], result of:
           0.037261043 = score(doc=312,freq=1.0), product of:
             0.11125768 = queryWeight, product of:
               1.4952936 = boost
               5.3585215 = idf(docFreq=536, maxDocs=41962)
               0.013885405 = queryNorm
             0.3349076 = fieldWeight in 312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3585215 = idf(docFreq=536, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.0390294 = weight(abstract_txt:experimental in 312) [ClassicSimilarity], result of:
           0.0390294 = score(doc=312,freq=1.0), product of:
             0.114750504 = queryWeight, product of:
               1.5185839 = boost
               5.441984 = idf(docFreq=493, maxDocs=41962)
               0.013885405 = queryNorm
             0.340124 = fieldWeight in 312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.441984 = idf(docFreq=493, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.06089476 = weight(abstract_txt:conditions in 312) [ClassicSimilarity], result of:
           0.06089476 = score(doc=312,freq=1.0), product of:
             0.15436421 = queryWeight, product of:
               1.7613059 = boost
               6.311801 = idf(docFreq=206, maxDocs=41962)
               0.013885405 = queryNorm
             0.39448756 = fieldWeight in 312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.311801 = idf(docFreq=206, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.05730778 = weight(abstract_txt:evaluation in 312) [ClassicSimilarity], result of:
           0.05730778 = score(doc=312,freq=3.0), product of:
             0.11765916 = queryWeight, product of:
               1.8833021 = boost
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.013885405 = queryNorm
             0.48706603 = fieldWeight in 312, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.05201015 = weight(abstract_txt:performance in 312) [ClassicSimilarity], result of:
           0.05201015 = score(doc=312,freq=2.0), product of:
             0.12625226 = queryWeight, product of:
               1.9508626 = boost
               4.66073 = idf(docFreq=1078, maxDocs=41962)
               0.013885405 = queryNorm
             0.4119542 = fieldWeight in 312, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.66073 = idf(docFreq=1078, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.0101974895 = weight(abstract_txt:that in 312) [ClassicSimilarity], result of:
           0.0101974895 = score(doc=312,freq=1.0), product of:
             0.06763881 = queryWeight, product of:
               2.0193894 = boost
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.013885405 = queryNorm
             0.15076388 = fieldWeight in 312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.04261008 = weight(abstract_txt:retrieval in 312) [ClassicSimilarity], result of:
           0.04261008 = score(doc=312,freq=2.0), product of:
             0.13927248 = queryWeight, product of:
               2.897709 = boost
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.013885405 = queryNorm
             0.30594757 = fieldWeight in 312, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.088246584 = weight(abstract_txt:relevance in 312) [ClassicSimilarity], result of:
           0.088246584 = score(doc=312,freq=1.0), product of:
             0.28510118 = queryWeight, product of:
               4.1459265 = boost
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.013885405 = queryNorm
             0.30952725 = fieldWeight in 312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
         0.22080754 = weight(abstract_txt:assessments in 312) [ClassicSimilarity], result of:
           0.22080754 = score(doc=312,freq=1.0), product of:
             0.4944806 = queryWeight, product of:
               4.984322 = boost
               7.14471 = idf(docFreq=89, maxDocs=41962)
               0.013885405 = queryNorm
             0.44654438 = fieldWeight in 312, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.14471 = idf(docFreq=89, maxDocs=41962)
               0.0625 = fieldNorm(doc=312)
       0.4 = coord(10/25)
   

3. Baillie, M.; Azzopardi, L.; Ruthven, I.: Evaluating epistemic uncertainty under incomplete assessments (2008) 0.25

   0.24518847 = sum of:
     0.24518847 = product of:
       0.766214 = sum of:
         0.042417817 = weight(abstract_txt:under in 4066) [ClassicSimilarity], result of:
           0.042417817 = score(doc=4066,freq=1.0), product of:
             0.104532704 = queryWeight, product of:
               1.4493977 = boost
               5.1940494 = idf(docFreq=632, maxDocs=41962)
               0.013885405 = queryNorm
             0.4057851 = fieldWeight in 4066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1940494 = idf(docFreq=632, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.071634725 = weight(abstract_txt:evaluation in 4066) [ClassicSimilarity], result of:
           0.071634725 = score(doc=4066,freq=3.0), product of:
             0.11765916 = queryWeight, product of:
               1.8833021 = boost
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.013885405 = queryNorm
             0.60883254 = fieldWeight in 4066, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.045970917 = weight(abstract_txt:performance in 4066) [ClassicSimilarity], result of:
           0.045970917 = score(doc=4066,freq=1.0), product of:
             0.12625226 = queryWeight, product of:
               1.9508626 = boost
               4.66073 = idf(docFreq=1078, maxDocs=41962)
               0.013885405 = queryNorm
             0.36411953 = fieldWeight in 4066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.66073 = idf(docFreq=1078, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.012746861 = weight(abstract_txt:that in 4066) [ClassicSimilarity], result of:
           0.012746861 = score(doc=4066,freq=1.0), product of:
             0.06763881 = queryWeight, product of:
               2.0193894 = boost
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.013885405 = queryNorm
             0.18845485 = fieldWeight in 4066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.053262595 = weight(abstract_txt:retrieval in 4066) [ClassicSimilarity], result of:
           0.053262595 = score(doc=4066,freq=2.0), product of:
             0.13927248 = queryWeight, product of:
               2.897709 = boost
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.013885405 = queryNorm
             0.38243446 = fieldWeight in 4066, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.1538635 = weight(abstract_txt:measurement in 4066) [ClassicSimilarity], result of:
           0.1538635 = score(doc=4066,freq=1.0), product of:
             0.28249288 = queryWeight, product of:
               2.918172 = boost
               6.9716907 = idf(docFreq=106, maxDocs=41962)
               0.013885405 = queryNorm
             0.5446633 = fieldWeight in 4066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9716907 = idf(docFreq=106, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.11030823 = weight(abstract_txt:relevance in 4066) [ClassicSimilarity], result of:
           0.11030823 = score(doc=4066,freq=1.0), product of:
             0.28510118 = queryWeight, product of:
               4.1459265 = boost
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.013885405 = queryNorm
             0.38690907 = fieldWeight in 4066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
         0.2760094 = weight(abstract_txt:assessments in 4066) [ClassicSimilarity], result of:
           0.2760094 = score(doc=4066,freq=1.0), product of:
             0.4944806 = queryWeight, product of:
               4.984322 = boost
               7.14471 = idf(docFreq=89, maxDocs=41962)
               0.013885405 = queryNorm
             0.55818045 = fieldWeight in 4066, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.14471 = idf(docFreq=89, maxDocs=41962)
               0.078125 = fieldNorm(doc=4066)
       0.32 = coord(8/25)
   

4. Lesk, M.E.; Salton, G.: Relevance assements and retrieval system evaluation (1969) 0.24

   0.24023196 = sum of:
     0.24023196 = product of:
       0.8579713 = sum of:
         0.06928725 = weight(abstract_txt:effectiveness in 4220) [ClassicSimilarity], result of:
           0.06928725 = score(doc=4220,freq=2.0), product of:
             0.101904035 = queryWeight, product of:
               1.4310578 = boost
               5.1283264 = idf(docFreq=675, maxDocs=41962)
               0.013885405 = queryNorm
             0.67992646 = fieldWeight in 4220, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1283264 = idf(docFreq=675, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
         0.058544103 = weight(abstract_txt:measures in 4220) [ClassicSimilarity], result of:
           0.058544103 = score(doc=4220,freq=1.0), product of:
             0.114750504 = queryWeight, product of:
               1.5185839 = boost
               5.441984 = idf(docFreq=493, maxDocs=41962)
               0.013885405 = queryNorm
             0.510186 = fieldWeight in 4220, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.441984 = idf(docFreq=493, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
         0.049629997 = weight(abstract_txt:evaluation in 4220) [ClassicSimilarity], result of:
           0.049629997 = score(doc=4220,freq=1.0), product of:
             0.11765916 = queryWeight, product of:
               1.8833021 = boost
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.013885405 = queryNorm
             0.42181158 = fieldWeight in 4220, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
         0.026493853 = weight(abstract_txt:that in 4220) [ClassicSimilarity], result of:
           0.026493853 = score(doc=4220,freq=3.0), product of:
             0.06763881 = queryWeight, product of:
               2.0193894 = boost
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.013885405 = queryNorm
             0.39169604 = fieldWeight in 4220, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
         0.045194812 = weight(abstract_txt:retrieval in 4220) [ClassicSimilarity], result of:
           0.045194812 = score(doc=4220,freq=1.0), product of:
             0.13927248 = queryWeight, product of:
               2.897709 = boost
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.013885405 = queryNorm
             0.3245064 = fieldWeight in 4220, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
         0.13236988 = weight(abstract_txt:relevance in 4220) [ClassicSimilarity], result of:
           0.13236988 = score(doc=4220,freq=1.0), product of:
             0.28510118 = queryWeight, product of:
               4.1459265 = boost
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.013885405 = queryNorm
             0.46429086 = fieldWeight in 4220, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
         0.47645146 = weight(abstract_txt:variations in 4220) [ClassicSimilarity], result of:
           0.47645146 = score(doc=4220,freq=2.0), product of:
             0.5314658 = queryWeight, product of:
               5.660565 = boost
               6.761718 = idf(docFreq=131, maxDocs=41962)
               0.013885405 = queryNorm
             0.89648557 = fieldWeight in 4220, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.761718 = idf(docFreq=131, maxDocs=41962)
               0.09375 = fieldNorm(doc=4220)
       0.28 = coord(7/25)
   

5. Ruthven, I.; Baillie, M.; Elsweiler, D.: ¬The relative effects of knowledge, interest and confidence in assessing relevance (2007) 0.22

   0.22034569 = sum of:
     0.22034569 = product of:
       0.61207134 = sum of:
         0.017667351 = weight(abstract_txt:what in 2836) [ClassicSimilarity], result of:
           0.017667351 = score(doc=2836,freq=1.0), product of:
             0.07394956 = queryWeight, product of:
               1.2190717 = boost
               4.368655 = idf(docFreq=1444, maxDocs=41962)
               0.013885405 = queryNorm
             0.23891082 = fieldWeight in 2836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.368655 = idf(docFreq=1444, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.034150723 = weight(abstract_txt:experimental in 2836) [ClassicSimilarity], result of:
           0.034150723 = score(doc=2836,freq=1.0), product of:
             0.114750504 = queryWeight, product of:
               1.5185839 = boost
               5.441984 = idf(docFreq=493, maxDocs=41962)
               0.013885405 = queryNorm
             0.2976085 = fieldWeight in 2836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.441984 = idf(docFreq=493, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.03833139 = weight(abstract_txt:effects in 2836) [ClassicSimilarity], result of:
           0.03833139 = score(doc=2836,freq=1.0), product of:
             0.12393416 = queryWeight, product of:
               1.5781816 = boost
               5.655558 = idf(docFreq=398, maxDocs=41962)
               0.013885405 = queryNorm
             0.30928832 = fieldWeight in 2836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.655558 = idf(docFreq=398, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.02895083 = weight(abstract_txt:evaluation in 2836) [ClassicSimilarity], result of:
           0.02895083 = score(doc=2836,freq=1.0), product of:
             0.11765916 = queryWeight, product of:
               1.8833021 = boost
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.013885405 = queryNorm
             0.24605675 = fieldWeight in 2836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4993234 = idf(docFreq=1267, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.019951994 = weight(abstract_txt:that in 2836) [ClassicSimilarity], result of:
           0.019951994 = score(doc=2836,freq=5.0), product of:
             0.06763881 = queryWeight, product of:
               2.0193894 = boost
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.013885405 = queryNorm
             0.2949785 = fieldWeight in 2836, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.4122221 = idf(docFreq=10221, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.11970723 = weight(abstract_txt:affect in 2836) [ClassicSimilarity], result of:
           0.11970723 = score(doc=2836,freq=3.0), product of:
             0.21016395 = queryWeight, product of:
               2.5170174 = boost
               6.013308 = idf(docFreq=278, maxDocs=41962)
               0.013885405 = queryNorm
             0.56958973 = fieldWeight in 2836, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.013308 = idf(docFreq=278, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.026363641 = weight(abstract_txt:retrieval in 2836) [ClassicSimilarity], result of:
           0.026363641 = score(doc=2836,freq=1.0), product of:
             0.13927248 = queryWeight, product of:
               2.897709 = boost
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.013885405 = queryNorm
             0.18929541 = fieldWeight in 2836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4614017 = idf(docFreq=3579, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.13374162 = weight(abstract_txt:relevance in 2836) [ClassicSimilarity], result of:
           0.13374162 = score(doc=2836,freq=3.0), product of:
             0.28510118 = queryWeight, product of:
               4.1459265 = boost
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.013885405 = queryNorm
             0.4691023 = fieldWeight in 2836, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.952436 = idf(docFreq=805, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
         0.1932066 = weight(abstract_txt:assessments in 2836) [ClassicSimilarity], result of:
           0.1932066 = score(doc=2836,freq=1.0), product of:
             0.4944806 = queryWeight, product of:
               4.984322 = boost
               7.14471 = idf(docFreq=89, maxDocs=41962)
               0.013885405 = queryNorm
             0.39072633 = fieldWeight in 2836, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.14471 = idf(docFreq=89, maxDocs=41962)
               0.0546875 = fieldNorm(doc=2836)
       0.36 = coord(9/25)