Document (#41459)

Author
Zhitomirsky-Geffet, M.
Bar-Ilan, J.
Levene, M.
Title
Categorical relevance judgment
Source
Journal of the Association for Information Science and Technology. 69(2018) no.9, S.1084-1094
Year
2018
Abstract
In this study we aim to explore users' behavior when assessing search results relevance based on the hypothesis of categorical thinking. To investigate how users categories search engine results, we perform several experiments where users are asked to group a list of 20 search results into several categories, while attaching a relevance judgment to each formed category. Moreover, to determine how users change their minds over time, each experiment was repeated three times under the same conditions, with a gap of one month between rounds. The results show that on average users form 4-5 categories. Within each round the size of a category decreases with the relevance of a category. To measure the agreement between the search engine's ranking and the users' relevance judgments, we defined two novel similarity measures, the average concordance and the MinMax swap ratio. Similarity is shown to be the highest for the third round as the users' opinion stabilizes. Qualitative analysis uncovered some interesting points that users tended to categories results by type and reliability of their source, and particularly, found commercial sites less trustworthy, and attached high relevance to Wikipedia when their prior domain knowledge was limited.
Content
Vgl.: https://onlinelibrary.wiley.com/doi/10.1002/asi.24035.

Similar documents (author)

  1. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Testing the stability of "wisdom of crowds" judgments of search results over time and their similarity with the search engine rankings (2016) 5.56
    5.564927 = sum of:
      5.564927 = sum of:
        1.0199649 = weight(author_txt:ilan in 5072) [ClassicSimilarity], result of:
          1.0199649 = score(doc=5072,freq=1.0), product of:
            0.4053526 = queryWeight, product of:
              8.051972 = idf(docFreq=36, maxDocs=42740)
              0.050342023 = queryNorm
            2.5162413 = fieldWeight in 5072, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.051972 = idf(docFreq=36, maxDocs=42740)
              0.3125 = fieldNorm(doc=5072)
        1.4713223 = weight(author_txt:zhitomirsky in 5072) [ClassicSimilarity], result of:
          1.4713223 = score(doc=5072,freq=1.0), product of:
            0.5175051 = queryWeight, product of:
              1.1299022 = boost
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.050342023 = queryNorm
            2.8431067 = fieldWeight in 5072, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.3125 = fieldNorm(doc=5072)
        1.4713223 = weight(author_txt:geffet in 5072) [ClassicSimilarity], result of:
          1.4713223 = score(doc=5072,freq=1.0), product of:
            0.5175051 = queryWeight, product of:
              1.1299022 = boost
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.050342023 = queryNorm
            2.8431067 = fieldWeight in 5072, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.3125 = fieldNorm(doc=5072)
        1.6023173 = weight(author_txt:levene in 5072) [ClassicSimilarity], result of:
          1.6023173 = score(doc=5072,freq=1.0), product of:
            0.5477829 = queryWeight, product of:
              1.1624861 = boost
              9.360306 = idf(docFreq=9, maxDocs=42740)
              0.050342023 = queryNorm
            2.9250956 = fieldWeight in 5072, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.360306 = idf(docFreq=9, maxDocs=42740)
              0.3125 = fieldNorm(doc=5072)
    
  2. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Analysis of change in users' assessment of search results over time (2017) 5.56
    5.564927 = sum of:
      5.564927 = sum of:
        1.0199649 = weight(author_txt:ilan in 5594) [ClassicSimilarity], result of:
          1.0199649 = score(doc=5594,freq=1.0), product of:
            0.4053526 = queryWeight, product of:
              8.051972 = idf(docFreq=36, maxDocs=42740)
              0.050342023 = queryNorm
            2.5162413 = fieldWeight in 5594, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.051972 = idf(docFreq=36, maxDocs=42740)
              0.3125 = fieldNorm(doc=5594)
        1.4713223 = weight(author_txt:zhitomirsky in 5594) [ClassicSimilarity], result of:
          1.4713223 = score(doc=5594,freq=1.0), product of:
            0.5175051 = queryWeight, product of:
              1.1299022 = boost
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.050342023 = queryNorm
            2.8431067 = fieldWeight in 5594, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.3125 = fieldNorm(doc=5594)
        1.4713223 = weight(author_txt:geffet in 5594) [ClassicSimilarity], result of:
          1.4713223 = score(doc=5594,freq=1.0), product of:
            0.5175051 = queryWeight, product of:
              1.1299022 = boost
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.050342023 = queryNorm
            2.8431067 = fieldWeight in 5594, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.097941 = idf(docFreq=12, maxDocs=42740)
              0.3125 = fieldNorm(doc=5594)
        1.6023173 = weight(author_txt:levene in 5594) [ClassicSimilarity], result of:
          1.6023173 = score(doc=5594,freq=1.0), product of:
            0.5477829 = queryWeight, product of:
              1.1624861 = boost
              9.360306 = idf(docFreq=9, maxDocs=42740)
              0.050342023 = queryNorm
            2.9250956 = fieldWeight in 5594, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.360306 = idf(docFreq=9, maxDocs=42740)
              0.3125 = fieldNorm(doc=5594)
    
  3. Zhitomirsky-Geffet, M.; Bar-Ilan, J.: Towards maximal unification of semantically diverse ontologies for controversial domains (2014) 3.57
    3.5663486 = sum of:
      3.5663486 = product of:
        4.7551312 = sum of:
          1.223958 = weight(author_txt:ilan in 3635) [ClassicSimilarity], result of:
            1.223958 = score(doc=3635,freq=1.0), product of:
              0.4053526 = queryWeight, product of:
                8.051972 = idf(docFreq=36, maxDocs=42740)
                0.050342023 = queryNorm
              3.0194898 = fieldWeight in 3635, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.051972 = idf(docFreq=36, maxDocs=42740)
                0.375 = fieldNorm(doc=3635)
          1.7655866 = weight(author_txt:zhitomirsky in 3635) [ClassicSimilarity], result of:
            1.7655866 = score(doc=3635,freq=1.0), product of:
              0.5175051 = queryWeight, product of:
                1.1299022 = boost
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.050342023 = queryNorm
              3.411728 = fieldWeight in 3635, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.375 = fieldNorm(doc=3635)
          1.7655866 = weight(author_txt:geffet in 3635) [ClassicSimilarity], result of:
            1.7655866 = score(doc=3635,freq=1.0), product of:
              0.5175051 = queryWeight, product of:
                1.1299022 = boost
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.050342023 = queryNorm
              3.411728 = fieldWeight in 3635, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.375 = fieldNorm(doc=3635)
        0.75 = coord(3/4)
    
  4. Bar-Ilan, J.; Zhitomirsky-Geffet, M.; Miller, Y.; Shoham, S.: ¬The effects of background information and social interaction on image tagging (2010) 2.97
    2.9719572 = sum of:
      2.9719572 = product of:
        3.9626095 = sum of:
          1.0199649 = weight(author_txt:ilan in 454) [ClassicSimilarity], result of:
            1.0199649 = score(doc=454,freq=1.0), product of:
              0.4053526 = queryWeight, product of:
                8.051972 = idf(docFreq=36, maxDocs=42740)
                0.050342023 = queryNorm
              2.5162413 = fieldWeight in 454, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.051972 = idf(docFreq=36, maxDocs=42740)
                0.3125 = fieldNorm(doc=454)
          1.4713223 = weight(author_txt:zhitomirsky in 454) [ClassicSimilarity], result of:
            1.4713223 = score(doc=454,freq=1.0), product of:
              0.5175051 = queryWeight, product of:
                1.1299022 = boost
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.050342023 = queryNorm
              2.8431067 = fieldWeight in 454, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.3125 = fieldNorm(doc=454)
          1.4713223 = weight(author_txt:geffet in 454) [ClassicSimilarity], result of:
            1.4713223 = score(doc=454,freq=1.0), product of:
              0.5175051 = queryWeight, product of:
                1.1299022 = boost
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.050342023 = queryNorm
              2.8431067 = fieldWeight in 454, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.3125 = fieldNorm(doc=454)
        0.75 = coord(3/4)
    
  5. Zhitomirsky-Geffet, M.; Erez, E.S.; Bar-Ilan, J.: Toward multiviewpoint ontology construction by collaboration of non-experts and crowdsourcing : the case of the effect of diet on health (2017) 2.97
    2.9719572 = sum of:
      2.9719572 = product of:
        3.9626095 = sum of:
          1.0199649 = weight(author_txt:ilan in 5440) [ClassicSimilarity], result of:
            1.0199649 = score(doc=5440,freq=1.0), product of:
              0.4053526 = queryWeight, product of:
                8.051972 = idf(docFreq=36, maxDocs=42740)
                0.050342023 = queryNorm
              2.5162413 = fieldWeight in 5440, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.051972 = idf(docFreq=36, maxDocs=42740)
                0.3125 = fieldNorm(doc=5440)
          1.4713223 = weight(author_txt:zhitomirsky in 5440) [ClassicSimilarity], result of:
            1.4713223 = score(doc=5440,freq=1.0), product of:
              0.5175051 = queryWeight, product of:
                1.1299022 = boost
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.050342023 = queryNorm
              2.8431067 = fieldWeight in 5440, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.3125 = fieldNorm(doc=5440)
          1.4713223 = weight(author_txt:geffet in 5440) [ClassicSimilarity], result of:
            1.4713223 = score(doc=5440,freq=1.0), product of:
              0.5175051 = queryWeight, product of:
                1.1299022 = boost
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.050342023 = queryNorm
              2.8431067 = fieldWeight in 5440, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.097941 = idf(docFreq=12, maxDocs=42740)
                0.3125 = fieldNorm(doc=5440)
        0.75 = coord(3/4)
    

Similar documents (content)

  1. Khoo, C.S.G.; Ng, K.; Ou, S.: ¬An exploratory study of human clustering of Web pages (2003) 0.26
    0.26337668 = sum of:
      0.26337668 = product of:
        0.65844166 = sum of:
          0.020520804 = weight(abstract_txt:several in 3742) [ClassicSimilarity], result of:
            0.020520804 = score(doc=3742,freq=1.0), product of:
              0.09603016 = queryWeight, product of:
                1.1670028 = boost
                4.5587463 = idf(docFreq=1216, maxDocs=42740)
                0.018050546 = queryNorm
              0.21369123 = fieldWeight in 3742, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.5587463 = idf(docFreq=1216, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.0149457725 = weight(abstract_txt:their in 3742) [ClassicSimilarity], result of:
            0.0149457725 = score(doc=3742,freq=2.0), product of:
              0.070628084 = queryWeight, product of:
                1.2257515 = boost
                3.1921601 = idf(docFreq=4772, maxDocs=42740)
                0.018050546 = queryNorm
              0.21161231 = fieldWeight in 3742, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.1921601 = idf(docFreq=4772, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.042558454 = weight(abstract_txt:similarity in 3742) [ClassicSimilarity], result of:
            0.042558454 = score(doc=3742,freq=1.0), product of:
              0.15617158 = queryWeight, product of:
                1.4882267 = boost
                5.8135657 = idf(docFreq=346, maxDocs=42740)
                0.018050546 = queryNorm
              0.2725109 = fieldWeight in 3742, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.8135657 = idf(docFreq=346, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.06386135 = weight(abstract_txt:average in 3742) [ClassicSimilarity], result of:
            0.06386135 = score(doc=3742,freq=2.0), product of:
              0.16246513 = queryWeight, product of:
                1.5179174 = boost
                5.929549 = idf(docFreq=308, maxDocs=42740)
                0.018050546 = queryNorm
              0.39307728 = fieldWeight in 3742, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.929549 = idf(docFreq=308, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.032437235 = weight(abstract_txt:each in 3742) [ClassicSimilarity], result of:
            0.032437235 = score(doc=3742,freq=2.0), product of:
              0.11839346 = queryWeight, product of:
                1.5870023 = boost
                4.132947 = idf(docFreq=1862, maxDocs=42740)
                0.018050546 = queryNorm
              0.27397826 = fieldWeight in 3742, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.132947 = idf(docFreq=1862, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.05965676 = weight(abstract_txt:search in 3742) [ClassicSimilarity], result of:
            0.05965676 = score(doc=3742,freq=8.0), product of:
              0.12322473 = queryWeight, product of:
                1.869528 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.018050546 = queryNorm
              0.4841298 = fieldWeight in 3742, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.033037715 = weight(abstract_txt:results in 3742) [ClassicSimilarity], result of:
            0.033037715 = score(doc=3742,freq=2.0), product of:
              0.14209804 = queryWeight, product of:
                2.2445645 = boost
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.018050546 = queryNorm
              0.23249945 = fieldWeight in 3742, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.19260313 = weight(abstract_txt:categories in 3742) [ClassicSimilarity], result of:
            0.19260313 = score(doc=3742,freq=10.0), product of:
              0.2498793 = queryWeight, product of:
                2.6622462 = boost
                5.199861 = idf(docFreq=640, maxDocs=42740)
                0.018050546 = queryNorm
              0.7707846 = fieldWeight in 3742, product of:
                3.1622777 = tf(freq=10.0), with freq of:
                  10.0 = termFreq=10.0
                5.199861 = idf(docFreq=640, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.08842249 = weight(abstract_txt:users in 3742) [ClassicSimilarity], result of:
            0.08842249 = score(doc=3742,freq=5.0), product of:
              0.23605683 = queryWeight, product of:
                3.6593702 = boost
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.018050546 = queryNorm
              0.37458137 = fieldWeight in 3742, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
          0.110397935 = weight(abstract_txt:relevance in 3742) [ClassicSimilarity], result of:
            0.110397935 = score(doc=3742,freq=2.0), product of:
              0.33750677 = queryWeight, product of:
                3.7893956 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.018050546 = queryNorm
              0.32709843 = fieldWeight in 3742, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.046875 = fieldNorm(doc=3742)
        0.4 = coord(10/25)
    
  2. Desai, M.; Spink, A.: ¬A algorithm to cluster documents based on relevance (2005) 0.19
    0.1850713 = sum of:
      0.1850713 = product of:
        0.66096896 = sum of:
          0.029618336 = weight(abstract_txt:when in 3036) [ClassicSimilarity], result of:
            0.029618336 = score(doc=3036,freq=2.0), product of:
              0.08035558 = queryWeight, product of:
                1.0675201 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.018050546 = queryNorm
              0.36859092 = fieldWeight in 3036, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
          0.030582117 = weight(abstract_txt:each in 3036) [ClassicSimilarity], result of:
            0.030582117 = score(doc=3036,freq=1.0), product of:
              0.11839346 = queryWeight, product of:
                1.5870023 = boost
                4.132947 = idf(docFreq=1862, maxDocs=42740)
                0.018050546 = queryNorm
              0.2583092 = fieldWeight in 3036, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.132947 = idf(docFreq=1862, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
          0.056244932 = weight(abstract_txt:search in 3036) [ClassicSimilarity], result of:
            0.056244932 = score(doc=3036,freq=4.0), product of:
              0.12322473 = queryWeight, product of:
                1.869528 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.018050546 = queryNorm
              0.45644194 = fieldWeight in 3036, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
          0.17641164 = weight(abstract_txt:categorical in 3036) [ClassicSimilarity], result of:
            0.17641164 = score(doc=3036,freq=1.0), product of:
              0.33266243 = queryWeight, product of:
                2.1720507 = boost
                8.484837 = idf(docFreq=23, maxDocs=42740)
                0.018050546 = queryNorm
              0.5303023 = fieldWeight in 3036, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.484837 = idf(docFreq=23, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
          0.044050287 = weight(abstract_txt:results in 3036) [ClassicSimilarity], result of:
            0.044050287 = score(doc=3036,freq=2.0), product of:
              0.14209804 = queryWeight, product of:
                2.2445645 = boost
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.018050546 = queryNorm
              0.30999926 = fieldWeight in 3036, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
          0.091322355 = weight(abstract_txt:users in 3036) [ClassicSimilarity], result of:
            0.091322355 = score(doc=3036,freq=3.0), product of:
              0.23605683 = queryWeight, product of:
                3.6593702 = boost
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.018050546 = queryNorm
              0.38686597 = fieldWeight in 3036, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
          0.2327393 = weight(abstract_txt:relevance in 3036) [ClassicSimilarity], result of:
            0.2327393 = score(doc=3036,freq=5.0), product of:
              0.33750677 = queryWeight, product of:
                3.7893956 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.018050546 = queryNorm
              0.6895841 = fieldWeight in 3036, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.0625 = fieldNorm(doc=3036)
        0.28 = coord(7/25)
    
  3. Balatsoukas, P.; Demian, P.: Effects of granularity of search results on the relevance judgment behavior of engineers : building systems for retrieval and understanding of context (2010) 0.18
    0.17782906 = sum of:
      0.17782906 = product of:
        0.63510376 = sum of:
          0.020943327 = weight(abstract_txt:when in 420) [ClassicSimilarity], result of:
            0.020943327 = score(doc=420,freq=1.0), product of:
              0.08035558 = queryWeight, product of:
                1.0675201 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.018050546 = queryNorm
              0.26063314 = fieldWeight in 420, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
          0.019927697 = weight(abstract_txt:their in 420) [ClassicSimilarity], result of:
            0.019927697 = score(doc=420,freq=2.0), product of:
              0.070628084 = queryWeight, product of:
                1.2257515 = boost
                3.1921601 = idf(docFreq=4772, maxDocs=42740)
                0.018050546 = queryNorm
              0.28214976 = fieldWeight in 420, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.1921601 = idf(docFreq=4772, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
          0.039771173 = weight(abstract_txt:search in 420) [ClassicSimilarity], result of:
            0.039771173 = score(doc=420,freq=2.0), product of:
              0.12322473 = queryWeight, product of:
                1.869528 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.018050546 = queryNorm
              0.3227532 = fieldWeight in 420, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
          0.22767855 = weight(abstract_txt:judgment in 420) [ClassicSimilarity], result of:
            0.22767855 = score(doc=420,freq=4.0), product of:
              0.24841644 = queryWeight, product of:
                1.8769739 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.018050546 = queryNorm
              0.91651964 = fieldWeight in 420, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
          0.044050287 = weight(abstract_txt:results in 420) [ClassicSimilarity], result of:
            0.044050287 = score(doc=420,freq=2.0), product of:
              0.14209804 = queryWeight, product of:
                2.2445645 = boost
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.018050546 = queryNorm
              0.30999926 = fieldWeight in 420, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
          0.07456439 = weight(abstract_txt:users in 420) [ClassicSimilarity], result of:
            0.07456439 = score(doc=420,freq=2.0), product of:
              0.23605683 = queryWeight, product of:
                3.6593702 = boost
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.018050546 = queryNorm
              0.31587473 = fieldWeight in 420, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
          0.20816834 = weight(abstract_txt:relevance in 420) [ClassicSimilarity], result of:
            0.20816834 = score(doc=420,freq=4.0), product of:
              0.33750677 = queryWeight, product of:
                3.7893956 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.018050546 = queryNorm
              0.6167827 = fieldWeight in 420, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.0625 = fieldNorm(doc=420)
        0.28 = coord(7/25)
    
  4. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Analysis of change in users' assessment of search results over time (2017) 0.18
    0.17620283 = sum of:
      0.17620283 = product of:
        0.6292958 = sum of:
          0.020943327 = weight(abstract_txt:when in 5594) [ClassicSimilarity], result of:
            0.020943327 = score(doc=5594,freq=1.0), product of:
              0.08035558 = queryWeight, product of:
                1.0675201 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.018050546 = queryNorm
              0.26063314 = fieldWeight in 5594, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
          0.014091009 = weight(abstract_txt:their in 5594) [ClassicSimilarity], result of:
            0.014091009 = score(doc=5594,freq=1.0), product of:
              0.070628084 = queryWeight, product of:
                1.2257515 = boost
                3.1921601 = idf(docFreq=4772, maxDocs=42740)
                0.018050546 = queryNorm
              0.19951001 = fieldWeight in 5594, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.1921601 = idf(docFreq=4772, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
          0.04870954 = weight(abstract_txt:search in 5594) [ClassicSimilarity], result of:
            0.04870954 = score(doc=5594,freq=3.0), product of:
              0.12322473 = queryWeight, product of:
                1.869528 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.018050546 = queryNorm
              0.39529032 = fieldWeight in 5594, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
          0.17641164 = weight(abstract_txt:categorical in 5594) [ClassicSimilarity], result of:
            0.17641164 = score(doc=5594,freq=1.0), product of:
              0.33266243 = queryWeight, product of:
                2.1720507 = boost
                8.484837 = idf(docFreq=23, maxDocs=42740)
                0.018050546 = queryNorm
              0.5303023 = fieldWeight in 5594, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.484837 = idf(docFreq=23, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
          0.06964962 = weight(abstract_txt:results in 5594) [ClassicSimilarity], result of:
            0.06964962 = score(doc=5594,freq=5.0), product of:
              0.14209804 = queryWeight, product of:
                2.2445645 = boost
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.018050546 = queryNorm
              0.49015188 = fieldWeight in 5594, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.5072412 = idf(docFreq=3482, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
          0.091322355 = weight(abstract_txt:users in 5594) [ClassicSimilarity], result of:
            0.091322355 = score(doc=5594,freq=3.0), product of:
              0.23605683 = queryWeight, product of:
                3.6593702 = boost
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.018050546 = queryNorm
              0.38686597 = fieldWeight in 5594, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
          0.20816834 = weight(abstract_txt:relevance in 5594) [ClassicSimilarity], result of:
            0.20816834 = score(doc=5594,freq=4.0), product of:
              0.33750677 = queryWeight, product of:
                3.7893956 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.018050546 = queryNorm
              0.6167827 = fieldWeight in 5594, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.0625 = fieldNorm(doc=5594)
        0.28 = coord(7/25)
    
  5. Xu, Y.: Relevance judgment in epistemic and hedonic information searches (2007) 0.17
    0.16909759 = sum of:
      0.16909759 = product of:
        0.84548795 = sum of:
          0.026179157 = weight(abstract_txt:when in 2065) [ClassicSimilarity], result of:
            0.026179157 = score(doc=2065,freq=1.0), product of:
              0.08035558 = queryWeight, product of:
                1.0675201 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.018050546 = queryNorm
              0.32579142 = fieldWeight in 2065, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.035153084 = weight(abstract_txt:search in 2065) [ClassicSimilarity], result of:
            0.035153084 = score(doc=2065,freq=1.0), product of:
              0.12322473 = queryWeight, product of:
                1.869528 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.018050546 = queryNorm
              0.2852762 = fieldWeight in 2065, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.24646926 = weight(abstract_txt:judgment in 2065) [ClassicSimilarity], result of:
            0.24646926 = score(doc=2065,freq=3.0), product of:
              0.24841644 = queryWeight, product of:
                1.8769739 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.018050546 = queryNorm
              0.99216163 = fieldWeight in 2065, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.14737082 = weight(abstract_txt:users in 2065) [ClassicSimilarity], result of:
            0.14737082 = score(doc=2065,freq=5.0), product of:
              0.23605683 = queryWeight, product of:
                3.6593702 = boost
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.018050546 = queryNorm
              0.62430227 = fieldWeight in 2065, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.5737147 = idf(docFreq=3258, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.39031568 = weight(abstract_txt:relevance in 2065) [ClassicSimilarity], result of:
            0.39031568 = score(doc=2065,freq=9.0), product of:
              0.33750677 = queryWeight, product of:
                3.7893956 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.018050546 = queryNorm
              1.1564677 = fieldWeight in 2065, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
        0.2 = coord(5/25)