Document (#39773)

Author
Ferro, N.
Silvello, G.
Keskustalo, H.
Pirkola, A.
Järvelin, K.
Title
¬The twist measure for IR evaluation : taking user's effort into account
Source
Journal of the Association for Information Science and Technology. 67(2016) no.3, S.620-648
Year
2016
Abstract
We present a novel measure for ranking evaluation, called Twist (t). It is a measure for informational intents, which handles both binary and graded relevance. t stems from the observation that searching is currently a that searching is currently taken for granted and it is natural for users to assume that search engines are available and work well. As a consequence, users may assume the utility they have in finding relevant documents, which is the focus of traditional measures, as granted. On the contrary, they may feel uneasy when the system returns nonrelevant documents because they are then forced to do additional work to get the desired information, and this causes avoidable effort. The latter is the focus of t, which evaluates the effectiveness of a system from the point of view of the effort required to the users to retrieve the desired information. We provide a formal definition of t, a demonstration of its properties, and introduce the notion of effort/gain plots, which complement traditional utility-based measures. By means of an extensive experimental evaluation, t is shown to grasp different aspects of system performances, to not require extensive and costly assessments, and to be a robust tool for detecting differences between systems.
Content
Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23416/abstract.

Similar documents (author)

  1. Pirkola, A.; Hedlund, T.; Keskustalo, H.; Järvelin, K.: Dictionary-based cross-language information retrieval : problems, methods, and research findings (2001) 2.13
    2.1330926 = sum of:
      2.1330926 = product of:
        3.5551543 = sum of:
          0.8043803 = weight(author_txt:järvelin in 4909) [ClassicSimilarity], result of:
            0.8043803 = score(doc=4909,freq=1.0), product of:
              0.3214735 = queryWeight, product of:
                8.006933 = idf(docFreq=37, maxDocs=41962)
                0.040149394 = queryNorm
              2.5021667 = fieldWeight in 4909, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.006933 = idf(docFreq=37, maxDocs=41962)
                0.3125 = fieldNorm(doc=4909)
          1.3212508 = weight(author_txt:pirkola in 4909) [ClassicSimilarity], result of:
            1.3212508 = score(doc=4909,freq=1.0), product of:
              0.44753578 = queryWeight, product of:
                1.1798893 = boost
                9.447295 = idf(docFreq=8, maxDocs=41962)
                0.040149394 = queryNorm
              2.9522798 = fieldWeight in 4909, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.447295 = idf(docFreq=8, maxDocs=41962)
                0.3125 = fieldNorm(doc=4909)
          1.4295232 = weight(author_txt:keskustalo in 4909) [ClassicSimilarity], result of:
            1.4295232 = score(doc=4909,freq=1.0), product of:
              0.4716629 = queryWeight, product of:
                1.2112764 = boost
                9.698609 = idf(docFreq=6, maxDocs=41962)
                0.040149394 = queryNorm
              3.0308154 = fieldWeight in 4909, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.698609 = idf(docFreq=6, maxDocs=41962)
                0.3125 = fieldNorm(doc=4909)
        0.6 = coord(3/5)
    
  2. Toivonen, J.; Pirkola, A.; Keskustalo, H.; Visala, K.; Järvelin, K.: Translating cross-lingual spelling variants using transformation rules (2005) 2.13
    2.1330926 = sum of:
      2.1330926 = product of:
        3.5551543 = sum of:
          0.8043803 = weight(author_txt:järvelin in 3053) [ClassicSimilarity], result of:
            0.8043803 = score(doc=3053,freq=1.0), product of:
              0.3214735 = queryWeight, product of:
                8.006933 = idf(docFreq=37, maxDocs=41962)
                0.040149394 = queryNorm
              2.5021667 = fieldWeight in 3053, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.006933 = idf(docFreq=37, maxDocs=41962)
                0.3125 = fieldNorm(doc=3053)
          1.3212508 = weight(author_txt:pirkola in 3053) [ClassicSimilarity], result of:
            1.3212508 = score(doc=3053,freq=1.0), product of:
              0.44753578 = queryWeight, product of:
                1.1798893 = boost
                9.447295 = idf(docFreq=8, maxDocs=41962)
                0.040149394 = queryNorm
              2.9522798 = fieldWeight in 3053, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.447295 = idf(docFreq=8, maxDocs=41962)
                0.3125 = fieldNorm(doc=3053)
          1.4295232 = weight(author_txt:keskustalo in 3053) [ClassicSimilarity], result of:
            1.4295232 = score(doc=3053,freq=1.0), product of:
              0.4716629 = queryWeight, product of:
                1.2112764 = boost
                9.698609 = idf(docFreq=6, maxDocs=41962)
                0.040149394 = queryNorm
              3.0308154 = fieldWeight in 3053, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.698609 = idf(docFreq=6, maxDocs=41962)
                0.3125 = fieldNorm(doc=3053)
        0.6 = coord(3/5)
    
  3. Ferro, N.; Silvello, G.: NESTOR: a formal model for digital archives (2013) 1.92
    1.9184326 = sum of:
      1.9184326 = product of:
        4.7960815 = sum of:
          2.3980408 = weight(author_txt:ferro in 4708) [ClassicSimilarity], result of:
            2.3980408 = score(doc=4708,freq=1.0), product of:
              0.48677543 = queryWeight, product of:
                1.2305286 = boost
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.040149394 = queryNorm
              4.92638 = fieldWeight in 4708, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.5 = fieldNorm(doc=4708)
          2.3980408 = weight(author_txt:silvello in 4708) [ClassicSimilarity], result of:
            2.3980408 = score(doc=4708,freq=1.0), product of:
              0.48677543 = queryWeight, product of:
                1.2305286 = boost
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.040149394 = queryNorm
              4.92638 = fieldWeight in 4708, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.5 = fieldNorm(doc=4708)
        0.4 = coord(2/5)
    
  4. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 1.92
    1.9184326 = sum of:
      1.9184326 = product of:
        4.7960815 = sum of:
          2.3980408 = weight(author_txt:ferro in 600) [ClassicSimilarity], result of:
            2.3980408 = score(doc=600,freq=1.0), product of:
              0.48677543 = queryWeight, product of:
                1.2305286 = boost
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.040149394 = queryNorm
              4.92638 = fieldWeight in 600, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.5 = fieldNorm(doc=600)
          2.3980408 = weight(author_txt:silvello in 600) [ClassicSimilarity], result of:
            2.3980408 = score(doc=600,freq=1.0), product of:
              0.48677543 = queryWeight, product of:
                1.2305286 = boost
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.040149394 = queryNorm
              4.92638 = fieldWeight in 600, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.85276 = idf(docFreq=5, maxDocs=41962)
                0.5 = fieldNorm(doc=600)
        0.4 = coord(2/5)
    
  5. Pirkola, A.; Järvelin, K.: Employing the resolution power of search keys (2001) 1.36
    1.3604039 = sum of:
      1.3604039 = product of:
        3.4010096 = sum of:
          1.2870084 = weight(author_txt:järvelin in 403) [ClassicSimilarity], result of:
            1.2870084 = score(doc=403,freq=1.0), product of:
              0.3214735 = queryWeight, product of:
                8.006933 = idf(docFreq=37, maxDocs=41962)
                0.040149394 = queryNorm
              4.0034666 = fieldWeight in 403, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.006933 = idf(docFreq=37, maxDocs=41962)
                0.5 = fieldNorm(doc=403)
          2.1140013 = weight(author_txt:pirkola in 403) [ClassicSimilarity], result of:
            2.1140013 = score(doc=403,freq=1.0), product of:
              0.44753578 = queryWeight, product of:
                1.1798893 = boost
                9.447295 = idf(docFreq=8, maxDocs=41962)
                0.040149394 = queryNorm
              4.7236476 = fieldWeight in 403, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.447295 = idf(docFreq=8, maxDocs=41962)
                0.5 = fieldNorm(doc=403)
        0.4 = coord(2/5)
    

Similar documents (content)

  1. Hert, C.A.; Nilan, M.S.: User-based information retrieval evaluation : an examination of an online public access catalog (1991) 0.14
    0.13900119 = sum of:
      0.13900119 = product of:
        0.49643284 = sum of:
          0.044083204 = weight(abstract_txt:searching in 3671) [ClassicSimilarity], result of:
            0.044083204 = score(doc=3671,freq=1.0), product of:
              0.09457247 = queryWeight, product of:
                1.0254118 = boost
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.021640936 = queryNorm
              0.46613145 = fieldWeight in 3671, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
          0.06912459 = weight(abstract_txt:focus in 3671) [ClassicSimilarity], result of:
            0.06912459 = score(doc=3671,freq=1.0), product of:
              0.12764515 = queryWeight, product of:
                1.1912918 = boost
                4.951196 = idf(docFreq=806, maxDocs=41962)
                0.021640936 = queryNorm
              0.5415371 = fieldWeight in 3671, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.951196 = idf(docFreq=806, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
          0.04595237 = weight(abstract_txt:system in 3671) [ClassicSimilarity], result of:
            0.04595237 = score(doc=3671,freq=2.0), product of:
              0.0883368 = queryWeight, product of:
                1.213759 = boost
                3.3630488 = idf(docFreq=3949, maxDocs=41962)
                0.021640936 = queryNorm
              0.52019507 = fieldWeight in 3671, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.3630488 = idf(docFreq=3949, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
          0.039192233 = weight(abstract_txt:users in 3671) [ClassicSimilarity], result of:
            0.039192233 = score(doc=3671,freq=1.0), product of:
              0.10009516 = queryWeight, product of:
                1.2920167 = boost
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.021640936 = queryNorm
              0.39154974 = fieldWeight in 3671, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
          0.15897709 = weight(abstract_txt:measures in 3671) [ClassicSimilarity], result of:
            0.15897709 = score(doc=3671,freq=3.0), product of:
              0.15420505 = queryWeight, product of:
                1.3093787 = boost
                5.441984 = idf(docFreq=493, maxDocs=41962)
                0.021640936 = queryNorm
              1.030946 = fieldWeight in 3671, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.441984 = idf(docFreq=493, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
          0.0290635 = weight(abstract_txt:which in 3671) [ClassicSimilarity], result of:
            0.0290635 = score(doc=3671,freq=1.0), product of:
              0.09025915 = queryWeight, product of:
                1.4166958 = boost
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.021640936 = queryNorm
              0.3220006 = fieldWeight in 3671, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
          0.11003985 = weight(abstract_txt:evaluation in 3671) [ClassicSimilarity], result of:
            0.11003985 = score(doc=3671,freq=2.0), product of:
              0.15811378 = queryWeight, product of:
                1.6238521 = boost
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.021640936 = queryNorm
              0.6959536 = fieldWeight in 3671, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.109375 = fieldNorm(doc=3671)
        0.28 = coord(7/25)
    
  2. Draper, S.W.; Dunlop, M.D.: New IR - new evaluation : the impact of interaction and multimedia on information retrieval and its evaluation (1997) 0.11
    0.11178882 = sum of:
      0.11178882 = product of:
        0.34934005 = sum of:
          0.03949976 = weight(abstract_txt:focus in 3463) [ClassicSimilarity], result of:
            0.03949976 = score(doc=3463,freq=1.0), product of:
              0.12764515 = queryWeight, product of:
                1.1912918 = boost
                4.951196 = idf(docFreq=806, maxDocs=41962)
                0.021640936 = queryNorm
              0.30944976 = fieldWeight in 3463, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.951196 = idf(docFreq=806, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.01856756 = weight(abstract_txt:system in 3463) [ClassicSimilarity], result of:
            0.01856756 = score(doc=3463,freq=1.0), product of:
              0.0883368 = queryWeight, product of:
                1.213759 = boost
                3.3630488 = idf(docFreq=3949, maxDocs=41962)
                0.021640936 = queryNorm
              0.21019055 = fieldWeight in 3463, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.3630488 = idf(docFreq=3949, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.038790252 = weight(abstract_txt:users in 3463) [ClassicSimilarity], result of:
            0.038790252 = score(doc=3463,freq=3.0), product of:
              0.10009516 = queryWeight, product of:
                1.2920167 = boost
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.021640936 = queryNorm
              0.38753372 = fieldWeight in 3463, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.052332107 = weight(abstract_txt:currently in 3463) [ClassicSimilarity], result of:
            0.052332107 = score(doc=3463,freq=1.0), product of:
              0.15397616 = queryWeight, product of:
                1.3084066 = boost
                5.437944 = idf(docFreq=495, maxDocs=41962)
                0.021640936 = queryNorm
              0.3398715 = fieldWeight in 3463, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.437944 = idf(docFreq=495, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.026783507 = weight(abstract_txt:they in 3463) [ClassicSimilarity], result of:
            0.026783507 = score(doc=3463,freq=1.0), product of:
              0.11277601 = queryWeight, product of:
                1.3714185 = boost
                3.7998872 = idf(docFreq=2551, maxDocs=41962)
                0.021640936 = queryNorm
              0.23749295 = fieldWeight in 3463, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.7998872 = idf(docFreq=2551, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.023486856 = weight(abstract_txt:which in 3463) [ClassicSimilarity], result of:
            0.023486856 = score(doc=3463,freq=2.0), product of:
              0.09025915 = queryWeight, product of:
                1.4166958 = boost
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.021640936 = queryNorm
              0.2602158 = fieldWeight in 3463, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.07286816 = weight(abstract_txt:extensive in 3463) [ClassicSimilarity], result of:
            0.07286816 = score(doc=3463,freq=1.0), product of:
              0.1919994 = queryWeight, product of:
                1.4610529 = boost
                6.0723658 = idf(docFreq=262, maxDocs=41962)
                0.021640936 = queryNorm
              0.37952286 = fieldWeight in 3463, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.0723658 = idf(docFreq=262, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
          0.07701185 = weight(abstract_txt:evaluation in 3463) [ClassicSimilarity], result of:
            0.07701185 = score(doc=3463,freq=3.0), product of:
              0.15811378 = queryWeight, product of:
                1.6238521 = boost
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.021640936 = queryNorm
              0.48706603 = fieldWeight in 3463, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.0625 = fieldNorm(doc=3463)
        0.32 = coord(8/25)
    
  3. Su, L.T.: ¬The relevance of recall and precision in user evaluation (1994) 0.11
    0.10855192 = sum of:
      0.10855192 = product of:
        0.38768542 = sum of:
          0.022041602 = weight(abstract_txt:searching in 6933) [ClassicSimilarity], result of:
            0.022041602 = score(doc=6933,freq=1.0), product of:
              0.09457247 = queryWeight, product of:
                1.0254118 = boost
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.021640936 = queryNorm
              0.23306572 = fieldWeight in 6933, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
          0.029054802 = weight(abstract_txt:traditional in 6933) [ClassicSimilarity], result of:
            0.029054802 = score(doc=6933,freq=1.0), product of:
              0.11369672 = queryWeight, product of:
                1.12432 = boost
                4.672851 = idf(docFreq=1065, maxDocs=41962)
                0.021640936 = queryNorm
              0.25554654 = fieldWeight in 6933, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.672851 = idf(docFreq=1065, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
          0.04800049 = weight(abstract_txt:users in 6933) [ClassicSimilarity], result of:
            0.04800049 = score(doc=6933,freq=6.0), product of:
              0.10009516 = queryWeight, product of:
                1.2920167 = boost
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.021640936 = queryNorm
              0.47954854 = fieldWeight in 6933, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
          0.1376782 = weight(abstract_txt:measures in 6933) [ClassicSimilarity], result of:
            0.1376782 = score(doc=6933,freq=9.0), product of:
              0.15420505 = queryWeight, product of:
                1.3093787 = boost
                5.441984 = idf(docFreq=493, maxDocs=41962)
                0.021640936 = queryNorm
              0.89282554 = fieldWeight in 6933, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                5.441984 = idf(docFreq=493, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
          0.01453175 = weight(abstract_txt:which in 6933) [ClassicSimilarity], result of:
            0.01453175 = score(doc=6933,freq=1.0), product of:
              0.09025915 = queryWeight, product of:
                1.4166958 = boost
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.021640936 = queryNorm
              0.1610003 = fieldWeight in 6933, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
          0.06738537 = weight(abstract_txt:evaluation in 6933) [ClassicSimilarity], result of:
            0.06738537 = score(doc=6933,freq=3.0), product of:
              0.15811378 = queryWeight, product of:
                1.6238521 = boost
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.021640936 = queryNorm
              0.42618278 = fieldWeight in 6933, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
          0.06899317 = weight(abstract_txt:measure in 6933) [ClassicSimilarity], result of:
            0.06899317 = score(doc=6933,freq=1.0), product of:
              0.23165257 = queryWeight, product of:
                1.9655323 = boost
                5.446041 = idf(docFreq=491, maxDocs=41962)
                0.021640936 = queryNorm
              0.29783037 = fieldWeight in 6933, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.446041 = idf(docFreq=491, maxDocs=41962)
                0.0546875 = fieldNorm(doc=6933)
        0.28 = coord(7/25)
    
  4. Vocht, L. De: Exploring semantic relationships in the Web of Data : Semantische relaties verkennen in data op het web (2017) 0.11
    0.107803255 = sum of:
      0.107803255 = product of:
        0.29945347 = sum of:
          0.019088587 = weight(abstract_txt:searching in 797) [ClassicSimilarity], result of:
            0.019088587 = score(doc=797,freq=3.0), product of:
              0.09457247 = queryWeight, product of:
                1.0254118 = boost
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.021640936 = queryNorm
              0.20184083 = fieldWeight in 797, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.06494422 = weight(abstract_txt:intents in 797) [ClassicSimilarity], result of:
            0.06494422 = score(doc=797,freq=1.0), product of:
              0.24489108 = queryWeight, product of:
                1.166776 = boost
                9.698609 = idf(docFreq=6, maxDocs=41962)
                0.021640936 = queryNorm
              0.26519635 = fieldWeight in 797, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.698609 = idf(docFreq=6, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.024439229 = weight(abstract_txt:focus in 797) [ClassicSimilarity], result of:
            0.024439229 = score(doc=797,freq=2.0), product of:
              0.12764515 = queryWeight, product of:
                1.1912918 = boost
                4.951196 = idf(docFreq=806, maxDocs=41962)
                0.021640936 = queryNorm
              0.19146226 = fieldWeight in 797, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.951196 = idf(docFreq=806, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.008123308 = weight(abstract_txt:system in 797) [ClassicSimilarity], result of:
            0.008123308 = score(doc=797,freq=1.0), product of:
              0.0883368 = queryWeight, product of:
                1.213759 = boost
                3.3630488 = idf(docFreq=3949, maxDocs=41962)
                0.021640936 = queryNorm
              0.09195837 = fieldWeight in 797, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.3630488 = idf(docFreq=3949, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.0353274 = weight(abstract_txt:users in 797) [ClassicSimilarity], result of:
            0.0353274 = score(doc=797,freq=13.0), product of:
              0.10009516 = queryWeight, product of:
                1.2920167 = boost
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.021640936 = queryNorm
              0.35293815 = fieldWeight in 797, product of:
                3.6055512 = tf(freq=13.0), with freq of:
                  13.0 = termFreq=13.0
                3.5798833 = idf(docFreq=3179, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.031002343 = weight(abstract_txt:they in 797) [ClassicSimilarity], result of:
            0.031002343 = score(doc=797,freq=7.0), product of:
              0.11277601 = queryWeight, product of:
                1.3714185 = boost
                3.7998872 = idf(docFreq=2551, maxDocs=41962)
                0.021640936 = queryNorm
              0.27490193 = fieldWeight in 797, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                3.7998872 = idf(docFreq=2551, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.01453175 = weight(abstract_txt:which in 797) [ClassicSimilarity], result of:
            0.01453175 = score(doc=797,freq=4.0), product of:
              0.09025915 = queryWeight, product of:
                1.4166958 = boost
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.021640936 = queryNorm
              0.1610003 = fieldWeight in 797, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                2.9440055 = idf(docFreq=6005, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.027509963 = weight(abstract_txt:evaluation in 797) [ClassicSimilarity], result of:
            0.027509963 = score(doc=797,freq=2.0), product of:
              0.15811378 = queryWeight, product of:
                1.6238521 = boost
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.021640936 = queryNorm
              0.1739884 = fieldWeight in 797, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
          0.074486665 = weight(abstract_txt:desired in 797) [ClassicSimilarity], result of:
            0.074486665 = score(doc=797,freq=2.0), product of:
              0.2683274 = queryWeight, product of:
                1.7272233 = boost
                7.1786118 = idf(docFreq=86, maxDocs=41962)
                0.021640936 = queryNorm
              0.2775962 = fieldWeight in 797, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.1786118 = idf(docFreq=86, maxDocs=41962)
                0.02734375 = fieldNorm(doc=797)
        0.36 = coord(9/25)
    
  5. Vakkari, P.; Huuskonen, S.: Search effort degrades search output but improves task outcome (2012) 0.10
    0.101798706 = sum of:
      0.101798706 = product of:
        0.5089935 = sum of:
          0.035624605 = weight(abstract_txt:searching in 2047) [ClassicSimilarity], result of:
            0.035624605 = score(doc=2047,freq=2.0), product of:
              0.09457247 = queryWeight, product of:
                1.0254118 = boost
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.021640936 = queryNorm
              0.37669107 = fieldWeight in 2047, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.261773 = idf(docFreq=1607, maxDocs=41962)
                0.0625 = fieldNorm(doc=2047)
          0.03320549 = weight(abstract_txt:traditional in 2047) [ClassicSimilarity], result of:
            0.03320549 = score(doc=2047,freq=1.0), product of:
              0.11369672 = queryWeight, product of:
                1.12432 = boost
                4.672851 = idf(docFreq=1065, maxDocs=41962)
                0.021640936 = queryNorm
              0.2920532 = fieldWeight in 2047, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.672851 = idf(docFreq=1065, maxDocs=41962)
                0.0625 = fieldNorm(doc=2047)
          0.07417386 = weight(abstract_txt:measures in 2047) [ClassicSimilarity], result of:
            0.07417386 = score(doc=2047,freq=2.0), product of:
              0.15420505 = queryWeight, product of:
                1.3093787 = boost
                5.441984 = idf(docFreq=493, maxDocs=41962)
                0.021640936 = queryNorm
              0.481008 = fieldWeight in 2047, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.441984 = idf(docFreq=493, maxDocs=41962)
                0.0625 = fieldNorm(doc=2047)
          0.06287991 = weight(abstract_txt:evaluation in 2047) [ClassicSimilarity], result of:
            0.06287991 = score(doc=2047,freq=2.0), product of:
              0.15811378 = queryWeight, product of:
                1.6238521 = boost
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.021640936 = queryNorm
              0.39768776 = fieldWeight in 2047, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4993234 = idf(docFreq=1267, maxDocs=41962)
                0.0625 = fieldNorm(doc=2047)
          0.30310962 = weight(abstract_txt:effort in 2047) [ClassicSimilarity], result of:
            0.30310962 = score(doc=2047,freq=6.0), product of:
              0.34432405 = queryWeight, product of:
                2.767035 = boost
                5.750117 = idf(docFreq=362, maxDocs=41962)
                0.021640936 = queryNorm
              0.8803033 = fieldWeight in 2047, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                5.750117 = idf(docFreq=362, maxDocs=41962)
                0.0625 = fieldNorm(doc=2047)
        0.2 = coord(5/25)