Document (#41458)

Author

Zhitomirsky-Geffet, M.
Bar-Ilan, J.
Levene, M.

Title

Categorical relevance judgment

Source

Journal of the Association for Information Science and Technology. 69(2018) no.9, S.1084-1094

Year

2018

Abstract

In this study we aim to explore users' behavior when assessing search results relevance based on the hypothesis of categorical thinking. To investigate how users categories search engine results, we perform several experiments where users are asked to group a list of 20 search results into several categories, while attaching a relevance judgment to each formed category. Moreover, to determine how users change their minds over time, each experiment was repeated three times under the same conditions, with a gap of one month between rounds. The results show that on average users form 4-5 categories. Within each round the size of a category decreases with the relevance of a category. To measure the agreement between the search engine's ranking and the users' relevance judgments, we defined two novel similarity measures, the average concordance and the MinMax swap ratio. Similarity is shown to be the highest for the third round as the users' opinion stabilizes. Qualitative analysis uncovered some interesting points that users tended to categories results by type and reliability of their source, and particularly, found commercial sites less trustworthy, and attached high relevance to Wikipedia when their prior domain knowledge was limited.

Content

Vgl.: https://onlinelibrary.wiley.com/doi/10.1002/asi.24035.

Similar documents (author)

1. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Testing the stability of "wisdom of crowds" judgments of search results over time and their similarity with the search engine rankings (2016) 5.50

   5.5016513 = sum of:
     5.5016513 = sum of:
       1.0574923 = weight(author_txt:ilan in 3071) [ClassicSimilarity], result of:
         1.0574923 = score(doc=3071,freq=1.0), product of:
           0.41849965 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.051756278 = queryNorm
           2.5268652 = fieldWeight in 3071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.3125 = fieldNorm(doc=3071)
       1.3929076 = weight(author_txt:geffet in 3071) [ClassicSimilarity], result of:
         1.3929076 = score(doc=3071,freq=1.0), product of:
           0.50287324 = queryWeight, product of:
             1.0961796 = boost
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.051756278 = queryNorm
           2.7698982 = fieldWeight in 3071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.3125 = fieldNorm(doc=3071)
       1.3929076 = weight(author_txt:zhitomirsky in 3071) [ClassicSimilarity], result of:
         1.3929076 = score(doc=3071,freq=1.0), product of:
           0.50287324 = queryWeight, product of:
             1.0961796 = boost
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.051756278 = queryNorm
           2.7698982 = fieldWeight in 3071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.3125 = fieldNorm(doc=3071)
       1.6583438 = weight(author_txt:levene in 3071) [ClassicSimilarity], result of:
         1.6583438 = score(doc=3071,freq=1.0), product of:
           0.56488496 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.051756278 = queryNorm
           2.9357195 = fieldWeight in 3071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.3125 = fieldNorm(doc=3071)
   

2. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Analysis of change in users' assessment of search results over time (2017) 5.50

   5.5016513 = sum of:
     5.5016513 = sum of:
       1.0574923 = weight(author_txt:ilan in 3593) [ClassicSimilarity], result of:
         1.0574923 = score(doc=3593,freq=1.0), product of:
           0.41849965 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.051756278 = queryNorm
           2.5268652 = fieldWeight in 3593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.3125 = fieldNorm(doc=3593)
       1.3929076 = weight(author_txt:geffet in 3593) [ClassicSimilarity], result of:
         1.3929076 = score(doc=3593,freq=1.0), product of:
           0.50287324 = queryWeight, product of:
             1.0961796 = boost
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.051756278 = queryNorm
           2.7698982 = fieldWeight in 3593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.3125 = fieldNorm(doc=3593)
       1.3929076 = weight(author_txt:zhitomirsky in 3593) [ClassicSimilarity], result of:
         1.3929076 = score(doc=3593,freq=1.0), product of:
           0.50287324 = queryWeight, product of:
             1.0961796 = boost
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.051756278 = queryNorm
           2.7698982 = fieldWeight in 3593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.863674 = idf(docFreq=16, maxDocs=44218)
             0.3125 = fieldNorm(doc=3593)
       1.6583438 = weight(author_txt:levene in 3593) [ClassicSimilarity], result of:
         1.6583438 = score(doc=3593,freq=1.0), product of:
           0.56488496 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.051756278 = queryNorm
           2.9357195 = fieldWeight in 3593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.3125 = fieldNorm(doc=3593)
   

3. Zhitomirsky-Geffet, M.; Bar-Ilan, J.: Towards maximal unification of semantically diverse ontologies for controversial domains (2014) 3.46

   3.4589767 = sum of:
     3.4589767 = product of:
       4.611969 = sum of:
         1.2689908 = weight(author_txt:ilan in 1634) [ClassicSimilarity], result of:
           1.2689908 = score(doc=1634,freq=1.0), product of:
             0.41849965 = queryWeight, product of:
               8.085969 = idf(docFreq=36, maxDocs=44218)
               0.051756278 = queryNorm
             3.0322385 = fieldWeight in 1634, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.085969 = idf(docFreq=36, maxDocs=44218)
               0.375 = fieldNorm(doc=1634)
         1.6714892 = weight(author_txt:geffet in 1634) [ClassicSimilarity], result of:
           1.6714892 = score(doc=1634,freq=1.0), product of:
             0.50287324 = queryWeight, product of:
               1.0961796 = boost
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.051756278 = queryNorm
             3.3238778 = fieldWeight in 1634, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.375 = fieldNorm(doc=1634)
         1.6714892 = weight(author_txt:zhitomirsky in 1634) [ClassicSimilarity], result of:
           1.6714892 = score(doc=1634,freq=1.0), product of:
             0.50287324 = queryWeight, product of:
               1.0961796 = boost
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.051756278 = queryNorm
             3.3238778 = fieldWeight in 1634, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.375 = fieldNorm(doc=1634)
       0.75 = coord(3/4)
   

4. Bar-Ilan, J.; Zhitomirsky-Geffet, M.; Miller, Y.; Shoham, S.: ¬The effects of background information and social interaction on image tagging (2010) 2.88

   2.8824806 = sum of:
     2.8824806 = product of:
       3.8433075 = sum of:
         1.0574923 = weight(author_txt:ilan in 3453) [ClassicSimilarity], result of:
           1.0574923 = score(doc=3453,freq=1.0), product of:
             0.41849965 = queryWeight, product of:
               8.085969 = idf(docFreq=36, maxDocs=44218)
               0.051756278 = queryNorm
             2.5268652 = fieldWeight in 3453, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.085969 = idf(docFreq=36, maxDocs=44218)
               0.3125 = fieldNorm(doc=3453)
         1.3929076 = weight(author_txt:geffet in 3453) [ClassicSimilarity], result of:
           1.3929076 = score(doc=3453,freq=1.0), product of:
             0.50287324 = queryWeight, product of:
               1.0961796 = boost
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.051756278 = queryNorm
             2.7698982 = fieldWeight in 3453, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.3125 = fieldNorm(doc=3453)
         1.3929076 = weight(author_txt:zhitomirsky in 3453) [ClassicSimilarity], result of:
           1.3929076 = score(doc=3453,freq=1.0), product of:
             0.50287324 = queryWeight, product of:
               1.0961796 = boost
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.051756278 = queryNorm
             2.7698982 = fieldWeight in 3453, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.3125 = fieldNorm(doc=3453)
       0.75 = coord(3/4)
   

5. Zhitomirsky-Geffet, M.; Erez, E.S.; Bar-Ilan, J.: Toward multiviewpoint ontology construction by collaboration of non-experts and crowdsourcing : the case of the effect of diet on health (2017) 2.88

   2.8824806 = sum of:
     2.8824806 = product of:
       3.8433075 = sum of:
         1.0574923 = weight(author_txt:ilan in 3439) [ClassicSimilarity], result of:
           1.0574923 = score(doc=3439,freq=1.0), product of:
             0.41849965 = queryWeight, product of:
               8.085969 = idf(docFreq=36, maxDocs=44218)
               0.051756278 = queryNorm
             2.5268652 = fieldWeight in 3439, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.085969 = idf(docFreq=36, maxDocs=44218)
               0.3125 = fieldNorm(doc=3439)
         1.3929076 = weight(author_txt:geffet in 3439) [ClassicSimilarity], result of:
           1.3929076 = score(doc=3439,freq=1.0), product of:
             0.50287324 = queryWeight, product of:
               1.0961796 = boost
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.051756278 = queryNorm
             2.7698982 = fieldWeight in 3439, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.3125 = fieldNorm(doc=3439)
         1.3929076 = weight(author_txt:zhitomirsky in 3439) [ClassicSimilarity], result of:
           1.3929076 = score(doc=3439,freq=1.0), product of:
             0.50287324 = queryWeight, product of:
               1.0961796 = boost
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.051756278 = queryNorm
             2.7698982 = fieldWeight in 3439, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.863674 = idf(docFreq=16, maxDocs=44218)
               0.3125 = fieldNorm(doc=3439)
       0.75 = coord(3/4)
   

Similar documents (content)

1. Khoo, C.S.G.; Ng, K.; Ou, S.: ¬An exploratory study of human clustering of Web pages (2003) 0.26

   0.26292992 = sum of:
     0.26292992 = product of:
       0.6573248 = sum of:
         0.020485045 = weight(abstract_txt:several in 2741) [ClassicSimilarity], result of:
           0.020485045 = score(doc=2741,freq=1.0), product of:
             0.096094504 = queryWeight, product of:
               1.1683583 = boost
               4.5477557 = idf(docFreq=1272, maxDocs=44218)
               0.018085284 = queryNorm
             0.21317604 = fieldWeight in 2741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5477557 = idf(docFreq=1272, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.01457158 = weight(abstract_txt:their in 2741) [ClassicSimilarity], result of:
           0.01457158 = score(doc=2741,freq=2.0), product of:
             0.069571614 = queryWeight, product of:
               1.2175547 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.018085284 = queryNorm
             0.2094472 = fieldWeight in 2741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.04291652 = weight(abstract_txt:similarity in 2741) [ClassicSimilarity], result of:
           0.04291652 = score(doc=2741,freq=1.0), product of:
             0.15733436 = queryWeight, product of:
               1.4949911 = boost
               5.8191514 = idf(docFreq=356, maxDocs=44218)
               0.018085284 = queryNorm
             0.27277273 = fieldWeight in 2741, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8191514 = idf(docFreq=356, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.063580036 = weight(abstract_txt:average in 2741) [ClassicSimilarity], result of:
           0.063580036 = score(doc=2741,freq=2.0), product of:
             0.16228476 = queryWeight, product of:
               1.5183283 = boost
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.018085284 = queryNorm
             0.39178067 = fieldWeight in 2741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.90999 = idf(docFreq=325, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.032280896 = weight(abstract_txt:each in 2741) [ClassicSimilarity], result of:
           0.032280896 = score(doc=2741,freq=2.0), product of:
             0.11822919 = queryWeight, product of:
               1.5872115 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.018085284 = queryNorm
             0.2730366 = fieldWeight in 2741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.06030736 = weight(abstract_txt:search in 2741) [ClassicSimilarity], result of:
           0.06030736 = score(doc=2741,freq=8.0), product of:
             0.12434666 = queryWeight, product of:
               1.8795716 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018085284 = queryNorm
             0.48499382 = fieldWeight in 2741, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.03251957 = weight(abstract_txt:results in 2741) [ClassicSimilarity], result of:
           0.03251957 = score(doc=2741,freq=2.0), product of:
             0.14086631 = queryWeight, product of:
               2.2366621 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018085284 = queryNorm
             0.23085415 = fieldWeight in 2741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.1912037 = weight(abstract_txt:categories in 2741) [ClassicSimilarity], result of:
           0.1912037 = score(doc=2741,freq=10.0), product of:
             0.24912354 = queryWeight, product of:
               2.6604137 = boost
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.018085284 = queryNorm
             0.7675055 = fieldWeight in 2741, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.088616416 = weight(abstract_txt:users in 2741) [ClassicSimilarity], result of:
           0.088616416 = score(doc=2741,freq=5.0), product of:
             0.23683542 = queryWeight, product of:
               3.6684291 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.018085284 = queryNorm
             0.37416875 = fieldWeight in 2741, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
         0.110843666 = weight(abstract_txt:relevance in 2741) [ClassicSimilarity], result of:
           0.110843666 = score(doc=2741,freq=2.0), product of:
             0.3390353 = queryWeight, product of:
               3.801105 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018085284 = queryNorm
             0.32693842 = fieldWeight in 2741, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.046875 = fieldNorm(doc=2741)
       0.4 = coord(10/25)
   

2. Desai, M.; Spink, A.: ¬A algorithm to cluster documents based on relevance (2005) 0.18

   0.18471122 = sum of:
     0.18471122 = product of:
       0.6596829 = sum of:
         0.029317005 = weight(abstract_txt:when in 1035) [ClassicSimilarity], result of:
           0.029317005 = score(doc=1035,freq=2.0), product of:
             0.07995602 = queryWeight, product of:
               1.0657427 = boost
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.018085284 = queryNorm
             0.36666414 = fieldWeight in 1035, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
         0.03043472 = weight(abstract_txt:each in 1035) [ClassicSimilarity], result of:
           0.03043472 = score(doc=1035,freq=1.0), product of:
             0.11822919 = queryWeight, product of:
               1.5872115 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.018085284 = queryNorm
             0.25742137 = fieldWeight in 1035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
         0.056858324 = weight(abstract_txt:search in 1035) [ClassicSimilarity], result of:
           0.056858324 = score(doc=1035,freq=4.0), product of:
             0.12434666 = queryWeight, product of:
               1.8795716 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018085284 = queryNorm
             0.45725656 = fieldWeight in 1035, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
         0.17451183 = weight(abstract_txt:categorical in 1035) [ClassicSimilarity], result of:
           0.17451183 = score(doc=1035,freq=1.0), product of:
             0.33087555 = queryWeight, product of:
               2.1679993 = boost
               8.43879 = idf(docFreq=25, maxDocs=44218)
               0.018085284 = queryNorm
             0.5274244 = fieldWeight in 1035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.43879 = idf(docFreq=25, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
         0.04335943 = weight(abstract_txt:results in 1035) [ClassicSimilarity], result of:
           0.04335943 = score(doc=1035,freq=2.0), product of:
             0.14086631 = queryWeight, product of:
               2.2366621 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018085284 = queryNorm
             0.30780554 = fieldWeight in 1035, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
         0.091522634 = weight(abstract_txt:users in 1035) [ClassicSimilarity], result of:
           0.091522634 = score(doc=1035,freq=3.0), product of:
             0.23683542 = queryWeight, product of:
               3.6684291 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.018085284 = queryNorm
             0.3864398 = fieldWeight in 1035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
         0.23367897 = weight(abstract_txt:relevance in 1035) [ClassicSimilarity], result of:
           0.23367897 = score(doc=1035,freq=5.0), product of:
             0.3390353 = queryWeight, product of:
               3.801105 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018085284 = queryNorm
             0.6892467 = fieldWeight in 1035, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=1035)
       0.28 = coord(7/25)
   

3. Balatsoukas, P.; Demian, P.: Effects of granularity of search results on the relevance judgment behavior of engineers : building systems for retrieval and understanding of context (2010) 0.18

   0.17840084 = sum of:
     0.17840084 = product of:
       0.6371459 = sum of:
         0.020730253 = weight(abstract_txt:when in 3419) [ClassicSimilarity], result of:
           0.020730253 = score(doc=3419,freq=1.0), product of:
             0.07995602 = queryWeight, product of:
               1.0657427 = boost
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.018085284 = queryNorm
             0.2592707 = fieldWeight in 3419, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
         0.019428773 = weight(abstract_txt:their in 3419) [ClassicSimilarity], result of:
           0.019428773 = score(doc=3419,freq=2.0), product of:
             0.069571614 = queryWeight, product of:
               1.2175547 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.018085284 = queryNorm
             0.27926293 = fieldWeight in 3419, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
         0.04020491 = weight(abstract_txt:search in 3419) [ClassicSimilarity], result of:
           0.04020491 = score(doc=3419,freq=2.0), product of:
             0.12434666 = queryWeight, product of:
               1.8795716 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018085284 = queryNorm
             0.3233292 = fieldWeight in 3419, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
         0.2296858 = weight(abstract_txt:judgment in 3419) [ClassicSimilarity], result of:
           0.2296858 = score(doc=3419,freq=4.0), product of:
             0.25033265 = queryWeight, product of:
               1.8857563 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.018085284 = queryNorm
             0.9175223 = fieldWeight in 3419, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
         0.04335943 = weight(abstract_txt:results in 3419) [ClassicSimilarity], result of:
           0.04335943 = score(doc=3419,freq=2.0), product of:
             0.14086631 = queryWeight, product of:
               2.2366621 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018085284 = queryNorm
             0.30780554 = fieldWeight in 3419, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
         0.074727915 = weight(abstract_txt:users in 3419) [ClassicSimilarity], result of:
           0.074727915 = score(doc=3419,freq=2.0), product of:
             0.23683542 = queryWeight, product of:
               3.6684291 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.018085284 = queryNorm
             0.31552678 = fieldWeight in 3419, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
         0.20900883 = weight(abstract_txt:relevance in 3419) [ClassicSimilarity], result of:
           0.20900883 = score(doc=3419,freq=4.0), product of:
             0.3390353 = queryWeight, product of:
               3.801105 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018085284 = queryNorm
             0.616481 = fieldWeight in 3419, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=3419)
       0.28 = coord(7/25)
   

4. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Analysis of change in users' assessment of search results over time (2017) 0.18

   0.17564674 = sum of:
     0.17564674 = product of:
       0.6273098 = sum of:
         0.020730253 = weight(abstract_txt:when in 3593) [ClassicSimilarity], result of:
           0.020730253 = score(doc=3593,freq=1.0), product of:
             0.07995602 = queryWeight, product of:
               1.0657427 = boost
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.018085284 = queryNorm
             0.2592707 = fieldWeight in 3593, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
         0.013738217 = weight(abstract_txt:their in 3593) [ClassicSimilarity], result of:
           0.013738217 = score(doc=3593,freq=1.0), product of:
             0.069571614 = queryWeight, product of:
               1.2175547 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.018085284 = queryNorm
             0.19746871 = fieldWeight in 3593, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
         0.049240753 = weight(abstract_txt:search in 3593) [ClassicSimilarity], result of:
           0.049240753 = score(doc=3593,freq=3.0), product of:
             0.12434666 = queryWeight, product of:
               1.8795716 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018085284 = queryNorm
             0.3959958 = fieldWeight in 3593, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
         0.17451183 = weight(abstract_txt:categorical in 3593) [ClassicSimilarity], result of:
           0.17451183 = score(doc=3593,freq=1.0), product of:
             0.33087555 = queryWeight, product of:
               2.1679993 = boost
               8.43879 = idf(docFreq=25, maxDocs=44218)
               0.018085284 = queryNorm
             0.5274244 = fieldWeight in 3593, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.43879 = idf(docFreq=25, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
         0.06855728 = weight(abstract_txt:results in 3593) [ClassicSimilarity], result of:
           0.06855728 = score(doc=3593,freq=5.0), product of:
             0.14086631 = queryWeight, product of:
               2.2366621 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018085284 = queryNorm
             0.48668328 = fieldWeight in 3593, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
         0.091522634 = weight(abstract_txt:users in 3593) [ClassicSimilarity], result of:
           0.091522634 = score(doc=3593,freq=3.0), product of:
             0.23683542 = queryWeight, product of:
               3.6684291 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.018085284 = queryNorm
             0.3864398 = fieldWeight in 3593, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
         0.20900883 = weight(abstract_txt:relevance in 3593) [ClassicSimilarity], result of:
           0.20900883 = score(doc=3593,freq=4.0), product of:
             0.3390353 = queryWeight, product of:
               3.801105 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018085284 = queryNorm
             0.616481 = fieldWeight in 3593, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=3593)
       0.28 = coord(7/25)
   

5. Xu, Y.: Relevance judgment in epistemic and hedonic information searches (2007) 0.17

   0.16993539 = sum of:
     0.16993539 = product of:
       0.84967697 = sum of:
         0.025912816 = weight(abstract_txt:when in 64) [ClassicSimilarity], result of:
           0.025912816 = score(doc=64,freq=1.0), product of:
             0.07995602 = queryWeight, product of:
               1.0657427 = boost
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.018085284 = queryNorm
             0.32408836 = fieldWeight in 64, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.148331 = idf(docFreq=1897, maxDocs=44218)
               0.078125 = fieldNorm(doc=64)
         0.035536453 = weight(abstract_txt:search in 64) [ClassicSimilarity], result of:
           0.035536453 = score(doc=64,freq=1.0), product of:
             0.12434666 = queryWeight, product of:
               1.8795716 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018085284 = queryNorm
             0.28578535 = fieldWeight in 64, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.078125 = fieldNorm(doc=64)
         0.24864216 = weight(abstract_txt:judgment in 64) [ClassicSimilarity], result of:
           0.24864216 = score(doc=64,freq=3.0), product of:
             0.25033265 = queryWeight, product of:
               1.8857563 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.018085284 = queryNorm
             0.99324703 = fieldWeight in 64, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.078125 = fieldNorm(doc=64)
         0.147694 = weight(abstract_txt:users in 64) [ClassicSimilarity], result of:
           0.147694 = score(doc=64,freq=5.0), product of:
             0.23683542 = queryWeight, product of:
               3.6684291 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.018085284 = queryNorm
             0.62361455 = fieldWeight in 64, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.078125 = fieldNorm(doc=64)
         0.39189154 = weight(abstract_txt:relevance in 64) [ClassicSimilarity], result of:
           0.39189154 = score(doc=64,freq=9.0), product of:
             0.3390353 = queryWeight, product of:
               3.801105 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018085284 = queryNorm
             1.1559019 = fieldWeight in 64, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=64)
       0.2 = coord(5/25)