Document (#40145)

Author

Schaer, P.
Mayr, P.
Sünkler, S.
Lewandowski, D.

Title

How relevant is the long tail? : a relevance assessment study on million short

Source

https://arxiv.org/abs/1606.06081

Year

2016

Abstract

Users of web search engines are known to mostly focus on the top ranked results of the search engine result page. While many studies support this well known information seeking pattern only few studies concentrate on the question what users are missing by neglecting lower ranked results. To learn more about the relevance distributions in the so-called long tail we conducted a relevance assessment study with the Million Short long-tail web search engine. While we see a clear difference in the content between the head and the tail of the search engine result list we see no statistical significant differences in the binary relevance judgments and weak significant differences when using graded relevance. The tail contains different but still valuable results. We argue that the long tail can be a rich source for the diversification of web search engine result lists but it needs more evaluation to clearly describe the differences.

Content

Die Studie wurde auf der diesjährigen CLEF-Konferenz mit dem Best Poster Award ausgezeichnet.

Footnote

To appear in Experimental IR Meets Multilinguality, Multimodality, and Interaction. 7th International Conference of the CLEF Association, CLEF 2016, \'Evora, Portugal, September 5-8, 2016.

Theme

Suchmaschinen
Retrievalstudien

Similar documents (author)

1. Lewandowski, D.; Sünkler, S.: ¬Das Relevance Assessment Tool : eine modulare Software zur Unterstützung bei der Durchführung vielfältiger Studien mit Suchmaschinen (2019) 2.08

   2.0816395 = sum of:
     2.0816395 = product of:
       4.163279 = sum of:
         1.5233063 = weight(author_txt:lewandowski in 5026) [ClassicSimilarity], result of:
           1.5233063 = score(doc=5026,freq=1.0), product of:
             0.40073597 = queryWeight, product of:
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.052710783 = queryNorm
             3.8012714 = fieldWeight in 5026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.5 = fieldNorm(doc=5026)
         2.6399727 = weight(author_txt:sünkler in 5026) [ClassicSimilarity], result of:
           2.6399727 = score(doc=5026,freq=1.0), product of:
             0.57818455 = queryWeight, product of:
               1.2011689 = boost
               9.131938 = idf(docFreq=12, maxDocs=44218)
               0.052710783 = queryNorm
             4.565969 = fieldWeight in 5026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.131938 = idf(docFreq=12, maxDocs=44218)
               0.5 = fieldNorm(doc=5026)
       0.5 = coord(2/4)
   

2. Lewandowski, D.; Sünkler, S.: What does Google recommend when you want to compare insurance offerings? (2019) 2.08

   2.0816395 = sum of:
     2.0816395 = product of:
       4.163279 = sum of:
         1.5233063 = weight(author_txt:lewandowski in 5288) [ClassicSimilarity], result of:
           1.5233063 = score(doc=5288,freq=1.0), product of:
             0.40073597 = queryWeight, product of:
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.052710783 = queryNorm
             3.8012714 = fieldWeight in 5288, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.5 = fieldNorm(doc=5288)
         2.6399727 = weight(author_txt:sünkler in 5288) [ClassicSimilarity], result of:
           2.6399727 = score(doc=5288,freq=1.0), product of:
             0.57818455 = queryWeight, product of:
               1.2011689 = boost
               9.131938 = idf(docFreq=12, maxDocs=44218)
               0.052710783 = queryNorm
             4.565969 = fieldWeight in 5288, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.131938 = idf(docFreq=12, maxDocs=44218)
               0.5 = fieldNorm(doc=5288)
       0.5 = coord(2/4)
   

3. Lewandowski, D.; Mayr, P.: Exploring the academic invisible Web (2006) 1.66

   1.6602535 = sum of:
     1.6602535 = product of:
       3.320507 = sum of:
         1.5233063 = weight(author_txt:lewandowski in 3752) [ClassicSimilarity], result of:
           1.5233063 = score(doc=3752,freq=1.0), product of:
             0.40073597 = queryWeight, product of:
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.052710783 = queryNorm
             3.8012714 = fieldWeight in 3752, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.5 = fieldNorm(doc=3752)
         1.7972009 = weight(author_txt:mayr in 3752) [ClassicSimilarity], result of:
           1.7972009 = score(doc=3752,freq=1.0), product of:
             0.44743636 = queryWeight, product of:
               1.0566629 = boost
               8.033325 = idf(docFreq=38, maxDocs=44218)
               0.052710783 = queryNorm
             4.0166626 = fieldWeight in 3752, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.033325 = idf(docFreq=38, maxDocs=44218)
               0.5 = fieldNorm(doc=3752)
       0.5 = coord(2/4)
   

4. Lewandowski, D.; Mayr, P.: Exploring the academic invisible Web (2006) 1.66

   1.6602535 = sum of:
     1.6602535 = product of:
       3.320507 = sum of:
         1.5233063 = weight(author_txt:lewandowski in 2580) [ClassicSimilarity], result of:
           1.5233063 = score(doc=2580,freq=1.0), product of:
             0.40073597 = queryWeight, product of:
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.052710783 = queryNorm
             3.8012714 = fieldWeight in 2580, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.602543 = idf(docFreq=59, maxDocs=44218)
               0.5 = fieldNorm(doc=2580)
         1.7972009 = weight(author_txt:mayr in 2580) [ClassicSimilarity], result of:
           1.7972009 = score(doc=2580,freq=1.0), product of:
             0.44743636 = queryWeight, product of:
               1.0566629 = boost
               8.033325 = idf(docFreq=38, maxDocs=44218)
               0.052710783 = queryNorm
             4.0166626 = fieldWeight in 2580, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.033325 = idf(docFreq=38, maxDocs=44218)
               0.5 = fieldNorm(doc=2580)
       0.5 = coord(2/4)
   

5. Mayr, P.; Schaer, P.; Mutschke, P.: ¬A science model driven retrieval prototype (2011) 1.60

   1.5979335 = sum of:
     1.5979335 = product of:
       3.195867 = sum of:
         1.3479006 = weight(author_txt:mayr in 649) [ClassicSimilarity], result of:
           1.3479006 = score(doc=649,freq=1.0), product of:
             0.44743636 = queryWeight, product of:
               1.0566629 = boost
               8.033325 = idf(docFreq=38, maxDocs=44218)
               0.052710783 = queryNorm
             3.012497 = fieldWeight in 649, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.033325 = idf(docFreq=38, maxDocs=44218)
               0.375 = fieldNorm(doc=649)
         1.8479664 = weight(author_txt:schaer in 649) [ClassicSimilarity], result of:
           1.8479664 = score(doc=649,freq=1.0), product of:
             0.55219024 = queryWeight, product of:
               1.173857 = boost
               8.924298 = idf(docFreq=15, maxDocs=44218)
               0.052710783 = queryNorm
             3.346612 = fieldWeight in 649, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.924298 = idf(docFreq=15, maxDocs=44218)
               0.375 = fieldNorm(doc=649)
       0.5 = coord(2/4)
   

Similar documents (content)

1. Li, X.; Schijvenaars, B.J.A.; Rijke, M.de: Investigating queries and search failures in academic search (2017) 0.22

   0.2188353 = sum of:
     0.2188353 = product of:
       0.6838603 = sum of:
         0.012304772 = weight(abstract_txt:users in 5033) [ClassicSimilarity], result of:
           0.012304772 = score(doc=5033,freq=2.0), product of:
             0.044568628 = queryWeight, product of:
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.012484986 = queryNorm
             0.27608594 = fieldWeight in 5033, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.020723904 = weight(abstract_txt:significant in 5033) [ClassicSimilarity], result of:
           0.020723904 = score(doc=5033,freq=1.0), product of:
             0.07948856 = queryWeight, product of:
               1.3354809 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.012484986 = queryNorm
             0.26071554 = fieldWeight in 5033, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.0270364 = weight(abstract_txt:known in 5033) [ClassicSimilarity], result of:
           0.0270364 = score(doc=5033,freq=1.0), product of:
             0.094905086 = queryWeight, product of:
               1.4592513 = boost
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.012484986 = queryNorm
             0.2848783 = fieldWeight in 5033, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.03508216 = weight(abstract_txt:differences in 5033) [ClassicSimilarity], result of:
           0.03508216 = score(doc=5033,freq=1.0), product of:
             0.12924434 = queryWeight, product of:
               2.0856276 = boost
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.012484986 = queryNorm
             0.2714406 = fieldWeight in 5033, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.038112596 = weight(abstract_txt:result in 5033) [ClassicSimilarity], result of:
           0.038112596 = score(doc=5033,freq=1.0), product of:
             0.13658395 = queryWeight, product of:
               2.1440299 = boost
               5.1024737 = idf(docFreq=730, maxDocs=44218)
               0.012484986 = queryNorm
             0.27904153 = fieldWeight in 5033, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1024737 = idf(docFreq=730, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.08757674 = weight(abstract_txt:search in 5033) [ClassicSimilarity], result of:
           0.08757674 = score(doc=5033,freq=14.0), product of:
             0.11700024 = queryWeight, product of:
               2.5618207 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.012484986 = queryNorm
             0.74851763 = fieldWeight in 5033, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.09135105 = weight(abstract_txt:engine in 5033) [ClassicSimilarity], result of:
           0.09135105 = score(doc=5033,freq=2.0), product of:
             0.21369742 = queryWeight, product of:
               3.0967062 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.012484986 = queryNorm
             0.42747846 = fieldWeight in 5033, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
         0.3716727 = weight(abstract_txt:tail in 5033) [ClassicSimilarity], result of:
           0.3716727 = score(doc=5033,freq=1.0), product of:
             0.7854848 = queryWeight, product of:
               7.271347 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012484986 = queryNorm
             0.47317618 = fieldWeight in 5033, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5033)
       0.32 = coord(8/25)
   

2. Crystal, A.; Greenberg, J.: Relevance criteria identified by health information users during Web searches (2006) 0.19

   0.19239484 = sum of:
     0.19239484 = product of:
       0.80164516 = sum of:
         0.031444922 = weight(abstract_txt:users in 5909) [ClassicSimilarity], result of:
           0.031444922 = score(doc=5909,freq=10.0), product of:
             0.044568628 = queryWeight, product of:
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.012484986 = queryNorm
             0.70553935 = fieldWeight in 5909, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.046331603 = weight(abstract_txt:search in 5909) [ClassicSimilarity], result of:
           0.046331603 = score(doc=5909,freq=3.0), product of:
             0.11700024 = queryWeight, product of:
               2.5618207 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.012484986 = queryNorm
             0.3959958 = fieldWeight in 5909, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.063591644 = weight(abstract_txt:long in 5909) [ClassicSimilarity], result of:
           0.063591644 = score(doc=5909,freq=1.0), product of:
             0.19346644 = queryWeight, product of:
               2.946478 = boost
               5.2591357 = idf(docFreq=624, maxDocs=44218)
               0.012484986 = queryNorm
             0.32869598 = fieldWeight in 5909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2591357 = idf(docFreq=624, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.10440119 = weight(abstract_txt:engine in 5909) [ClassicSimilarity], result of:
           0.10440119 = score(doc=5909,freq=2.0), product of:
             0.21369742 = queryWeight, product of:
               3.0967062 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.012484986 = queryNorm
             0.48854682 = fieldWeight in 5909, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.13110703 = weight(abstract_txt:relevance in 5909) [ClassicSimilarity], result of:
           0.13110703 = score(doc=5909,freq=4.0), product of:
             0.21267003 = queryWeight, product of:
               3.45389 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012484986 = queryNorm
             0.616481 = fieldWeight in 5909, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.4247688 = weight(abstract_txt:tail in 5909) [ClassicSimilarity], result of:
           0.4247688 = score(doc=5909,freq=1.0), product of:
             0.7854848 = queryWeight, product of:
               7.271347 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012484986 = queryNorm
             0.5407728 = fieldWeight in 5909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
       0.24 = coord(6/25)
   

3. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Testing the stability of "wisdom of crowds" judgments of search results over time and their similarity with the search engine rankings (2016) 0.17

   0.16988595 = sum of:
     0.16988595 = product of:
       0.42471486 = sum of:
         0.015070206 = weight(abstract_txt:users in 3071) [ClassicSimilarity], result of:
           0.015070206 = score(doc=3071,freq=3.0), product of:
             0.044568628 = queryWeight, product of:
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.012484986 = queryNorm
             0.33813483 = fieldWeight in 3071, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.014869489 = weight(abstract_txt:studies in 3071) [ClassicSimilarity], result of:
           0.014869489 = score(doc=3071,freq=1.0), product of:
             0.06370707 = queryWeight, product of:
               1.1955814 = boost
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.012484986 = queryNorm
             0.23340407 = fieldWeight in 3071, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.038235243 = weight(abstract_txt:known in 3071) [ClassicSimilarity], result of:
           0.038235243 = score(doc=3071,freq=2.0), product of:
             0.094905086 = queryWeight, product of:
               1.4592513 = boost
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.012484986 = queryNorm
             0.40287876 = fieldWeight in 3071, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.020986067 = weight(abstract_txt:results in 3071) [ClassicSimilarity], result of:
           0.020986067 = score(doc=3071,freq=3.0), product of:
             0.06362107 = queryWeight, product of:
               1.4632936 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.012484986 = queryNorm
             0.32986036 = fieldWeight in 3071, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.03648878 = weight(abstract_txt:assessment in 3071) [ClassicSimilarity], result of:
           0.03648878 = score(doc=3071,freq=1.0), product of:
             0.115903474 = queryWeight, product of:
               1.6126257 = boost
               5.756716 = idf(docFreq=379, maxDocs=44218)
               0.012484986 = queryNorm
             0.3148204 = fieldWeight in 3071, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.756716 = idf(docFreq=379, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.03508216 = weight(abstract_txt:differences in 3071) [ClassicSimilarity], result of:
           0.03508216 = score(doc=3071,freq=1.0), product of:
             0.12924434 = queryWeight, product of:
               2.0856276 = boost
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.012484986 = queryNorm
             0.2714406 = fieldWeight in 3071, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.038112596 = weight(abstract_txt:result in 3071) [ClassicSimilarity], result of:
           0.038112596 = score(doc=3071,freq=1.0), product of:
             0.13658395 = queryWeight, product of:
               2.1440299 = boost
               5.1024737 = idf(docFreq=730, maxDocs=44218)
               0.012484986 = queryNorm
             0.27904153 = fieldWeight in 3071, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.1024737 = idf(docFreq=730, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.061926104 = weight(abstract_txt:search in 3071) [ClassicSimilarity], result of:
           0.061926104 = score(doc=3071,freq=7.0), product of:
             0.11700024 = queryWeight, product of:
               2.5618207 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.012484986 = queryNorm
             0.52928185 = fieldWeight in 3071, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.06459495 = weight(abstract_txt:engine in 3071) [ClassicSimilarity], result of:
           0.06459495 = score(doc=3071,freq=1.0), product of:
             0.21369742 = queryWeight, product of:
               3.0967062 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.012484986 = queryNorm
             0.30227295 = fieldWeight in 3071, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
         0.09934927 = weight(abstract_txt:relevance in 3071) [ClassicSimilarity], result of:
           0.09934927 = score(doc=3071,freq=3.0), product of:
             0.21267003 = queryWeight, product of:
               3.45389 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012484986 = queryNorm
             0.46715218 = fieldWeight in 3071, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3071)
       0.4 = coord(10/25)
   

4. Peters, I.; Stock, W.G.: Power tags in information retrieval (2010) 0.15

   0.14689453 = sum of:
     0.14689453 = product of:
       0.61206055 = sum of:
         0.019140309 = weight(abstract_txt:while in 865) [ClassicSimilarity], result of:
           0.019140309 = score(doc=865,freq=1.0), product of:
             0.06896492 = queryWeight, product of:
               1.2439401 = boost
               4.44059 = idf(docFreq=1416, maxDocs=44218)
               0.012484986 = queryNorm
             0.27753687 = fieldWeight in 865, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.44059 = idf(docFreq=1416, maxDocs=44218)
               0.0625 = fieldNorm(doc=865)
         0.030898742 = weight(abstract_txt:known in 865) [ClassicSimilarity], result of:
           0.030898742 = score(doc=865,freq=1.0), product of:
             0.094905086 = queryWeight, product of:
               1.4592513 = boost
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.012484986 = queryNorm
             0.3255752 = fieldWeight in 865, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2092032 = idf(docFreq=656, maxDocs=44218)
               0.0625 = fieldNorm(doc=865)
         0.013847213 = weight(abstract_txt:results in 865) [ClassicSimilarity], result of:
           0.013847213 = score(doc=865,freq=1.0), product of:
             0.06362107 = queryWeight, product of:
               1.4632936 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.012484986 = queryNorm
             0.21765138 = fieldWeight in 865, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=865)
         0.059813842 = weight(abstract_txt:search in 865) [ClassicSimilarity], result of:
           0.059813842 = score(doc=865,freq=5.0), product of:
             0.11700024 = queryWeight, product of:
               2.5618207 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.012484986 = queryNorm
             0.5112284 = fieldWeight in 865, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=865)
         0.063591644 = weight(abstract_txt:long in 865) [ClassicSimilarity], result of:
           0.063591644 = score(doc=865,freq=1.0), product of:
             0.19346644 = queryWeight, product of:
               2.946478 = boost
               5.2591357 = idf(docFreq=624, maxDocs=44218)
               0.012484986 = queryNorm
             0.32869598 = fieldWeight in 865, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2591357 = idf(docFreq=624, maxDocs=44218)
               0.0625 = fieldNorm(doc=865)
         0.4247688 = weight(abstract_txt:tail in 865) [ClassicSimilarity], result of:
           0.4247688 = score(doc=865,freq=1.0), product of:
             0.7854848 = queryWeight, product of:
               7.271347 = boost
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.012484986 = queryNorm
             0.5407728 = fieldWeight in 865, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.652365 = idf(docFreq=20, maxDocs=44218)
               0.0625 = fieldNorm(doc=865)
       0.24 = coord(6/25)
   

5. Can, F.; Nuray, R.; Sevdik, A.B.: Automatic performance evaluation of Web search engines (2004) 0.15

   0.14551564 = sum of:
     0.14551564 = product of:
       0.45473635 = sum of:
         0.0124296965 = weight(abstract_txt:users in 2570) [ClassicSimilarity], result of:
           0.0124296965 = score(doc=2570,freq=1.0), product of:
             0.044568628 = queryWeight, product of:
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.012484986 = queryNorm
             0.2788889 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.053199872 = weight(abstract_txt:binary in 2570) [ClassicSimilarity], result of:
           0.053199872 = score(doc=2570,freq=1.0), product of:
             0.09325082 = queryWeight, product of:
               1.022814 = boost
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.012484986 = queryNorm
             0.570503 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.029605579 = weight(abstract_txt:significant in 2570) [ClassicSimilarity], result of:
           0.029605579 = score(doc=2570,freq=1.0), product of:
             0.07948856 = queryWeight, product of:
               1.3354809 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.012484986 = queryNorm
             0.3724508 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.017309017 = weight(abstract_txt:results in 2570) [ClassicSimilarity], result of:
           0.017309017 = score(doc=2570,freq=1.0), product of:
             0.06362107 = queryWeight, product of:
               1.4632936 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.012484986 = queryNorm
             0.27206424 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.052126825 = weight(abstract_txt:assessment in 2570) [ClassicSimilarity], result of:
           0.052126825 = score(doc=2570,freq=1.0), product of:
             0.115903474 = queryWeight, product of:
               1.6126257 = boost
               5.756716 = idf(docFreq=379, maxDocs=44218)
               0.012484986 = queryNorm
             0.44974342 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.756716 = idf(docFreq=379, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.08190348 = weight(abstract_txt:search in 2570) [ClassicSimilarity], result of:
           0.08190348 = score(doc=2570,freq=6.0), product of:
             0.11700024 = queryWeight, product of:
               2.5618207 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.012484986 = queryNorm
             0.7000283 = fieldWeight in 2570, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.092278495 = weight(abstract_txt:engine in 2570) [ClassicSimilarity], result of:
           0.092278495 = score(doc=2570,freq=1.0), product of:
             0.21369742 = queryWeight, product of:
               3.0967062 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.012484986 = queryNorm
             0.4318185 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.11588334 = weight(abstract_txt:relevance in 2570) [ClassicSimilarity], result of:
           0.11588334 = score(doc=2570,freq=2.0), product of:
             0.21267003 = queryWeight, product of:
               3.45389 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012484986 = queryNorm
             0.5448974 = fieldWeight in 2570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
       0.32 = coord(8/25)