Document (#27118)

Author

Su, L.T.

Title

¬A comprehensive and systematic model of user evaluation of Web search engines : Il. An evaluation by undergraduates

Source

Journal of the American Society for Information Science and technology. 54(2003) no.13, S.1193-1222

Year

2003

Abstract

This paper presents an application of the model described in Part I to the evaluation of Web search engines by undergraduates. The study observed how 36 undergraduate used four major search engines to find information for their own individual problems and how they evaluated these engines based an actual interaction with the search engines. User evaluation was based an 16 performance measures representing five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Non-performance (user-related) measures were also applied. Each participant searched his/ her own topic an all four engines and provided satisfaction ratings for system features and interaction and reasons for satisfaction. Each also made relevance judgements of retrieved items in relation to his/her own information need and participated in post-search Interviews to provide reactions to the search results and overall performance. The study found significant differences in precision PR1 relative recall, user satisfaction with output display, time saving, value of search results, and overall performance among the four engines and also significant engine by discipline interactions an all these measures. In addition, the study found significant differences in user satisfaction with response time among four engines, and significant engine by discipline interaction in user satisfaction with search interface. None of the four search engines dominated in every aspect of the multidimensional evaluation. Content analysis of verbal data identified a number of user criteria and users evaluative comments based an these criteria. Results from both quantitative analysis and content analysis provide insight for system design and development, and useful feedback an strengths and weaknesses of search engines for system improvement

Footnote

Teil I im selben Heft

Theme

Suchmaschinen
Benutzerstudien

Similar documents (content)

1. Su, L.T.; Chen, H.L.: Evaluation of Web search engines by undergraduate students (1999) 1.25

   1.2452863 = sum of:
     1.2452863 = product of:
       1.5566078 = sum of:
         0.01415742 = weight(abstract_txt:based in 6546) [ClassicSimilarity], result of:
           0.01415742 = score(doc=6546,freq=2.0), product of:
             0.057421245 = queryWeight, product of:
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.018012092 = queryNorm
             0.24655369 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.014160107 = weight(abstract_txt:these in 6546) [ClassicSimilarity], result of:
           0.014160107 = score(doc=6546,freq=2.0), product of:
             0.05742851 = queryWeight, product of:
               1.0000632 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.018012092 = queryNorm
             0.24656929 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.04279984 = weight(abstract_txt:relevance in 6546) [ClassicSimilarity], result of:
           0.04279984 = score(doc=6546,freq=3.0), product of:
             0.09161863 = queryWeight, product of:
               1.0313592 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018012092 = queryNorm
             0.46715218 = fieldWeight in 6546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.0251891 = weight(abstract_txt:differences in 6546) [ClassicSimilarity], result of:
           0.0251891 = score(doc=6546,freq=1.0), product of:
             0.09279784 = queryWeight, product of:
               1.0379752 = boost
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.018012092 = queryNorm
             0.2714406 = fieldWeight in 6546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.016758712 = weight(abstract_txt:system in 6546) [ClassicSimilarity], result of:
           0.016758712 = score(doc=6546,freq=2.0), product of:
             0.06425549 = queryWeight, product of:
               1.0578371 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.018012092 = queryNorm
             0.26081368 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.017538248 = weight(abstract_txt:study in 6546) [ClassicSimilarity], result of:
           0.017538248 = score(doc=6546,freq=2.0), product of:
             0.06623293 = queryWeight, product of:
               1.0739911 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.018012092 = queryNorm
             0.2647965 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.01845453 = weight(abstract_txt:results in 6546) [ClassicSimilarity], result of:
           0.01845453 = score(doc=6546,freq=2.0), product of:
             0.06852018 = queryWeight, product of:
               1.092378 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018012092 = queryNorm
             0.26932985 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.03386779 = weight(abstract_txt:overall in 6546) [ClassicSimilarity], result of:
           0.03386779 = score(doc=6546,freq=1.0), product of:
             0.11304569 = queryWeight, product of:
               1.1456318 = boost
               5.478287 = idf(docFreq=501, maxDocs=44218)
               0.018012092 = queryNorm
             0.29959384 = fieldWeight in 6546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.478287 = idf(docFreq=501, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.04919274 = weight(abstract_txt:engine in 6546) [ClassicSimilarity], result of:
           0.04919274 = score(doc=6546,freq=2.0), product of:
             0.115076534 = queryWeight, product of:
               1.1558765 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.018012092 = queryNorm
             0.42747846 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.044822894 = weight(abstract_txt:interaction in 6546) [ClassicSimilarity], result of:
           0.044822894 = score(doc=6546,freq=1.0), product of:
             0.1559887 = queryWeight, product of:
               1.6482013 = boost
               5.254347 = idf(docFreq=627, maxDocs=44218)
               0.018012092 = queryNorm
             0.28734708 = fieldWeight in 6546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.254347 = idf(docFreq=627, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.08594049 = weight(abstract_txt:measures in 6546) [ClassicSimilarity], result of:
           0.08594049 = score(doc=6546,freq=3.0), product of:
             0.16692369 = queryWeight, product of:
               1.7049932 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.018012092 = queryNorm
             0.51484895 = fieldWeight in 6546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.05352255 = weight(abstract_txt:criteria in 6546) [ClassicSimilarity], result of:
           0.05352255 = score(doc=6546,freq=1.0), product of:
             0.1755703 = queryWeight, product of:
               1.7485949 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.018012092 = queryNorm
             0.30484968 = fieldWeight in 6546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.05784405 = weight(abstract_txt:performance in 6546) [ClassicSimilarity], result of:
           0.05784405 = score(doc=6546,freq=2.0), product of:
             0.16152331 = queryWeight, product of:
               1.9366478 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.018012092 = queryNorm
             0.3581158 = fieldWeight in 6546, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.044639472 = weight(abstract_txt:significant in 6546) [ClassicSimilarity], result of:
           0.044639472 = score(doc=6546,freq=1.0), product of:
             0.17121907 = queryWeight, product of:
               1.9939264 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.018012092 = queryNorm
             0.26071554 = fieldWeight in 6546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.14407925 = weight(abstract_txt:four in 6546) [ClassicSimilarity], result of:
           0.14407925 = score(doc=6546,freq=4.0), product of:
             0.2537603 = queryWeight, product of:
               2.713938 = boost
               5.191103 = idf(docFreq=668, maxDocs=44218)
               0.018012092 = queryNorm
             0.5677769 = fieldWeight in 6546, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.191103 = idf(docFreq=668, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.09663448 = weight(abstract_txt:evaluation in 6546) [ClassicSimilarity], result of:
           0.09663448 = score(doc=6546,freq=3.0), product of:
             0.22741416 = queryWeight, product of:
               2.8144104 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.018012092 = queryNorm
             0.42492726 = fieldWeight in 6546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.041182254 = weight(abstract_txt:user in 6546) [ClassicSimilarity], result of:
           0.041182254 = score(doc=6546,freq=1.0), product of:
             0.20443542 = queryWeight, product of:
               3.0812442 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.018012092 = queryNorm
             0.20144382 = fieldWeight in 6546, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.087323844 = weight(abstract_txt:search in 6546) [ClassicSimilarity], result of:
           0.087323844 = score(doc=6546,freq=3.0), product of:
             0.25201955 = queryWeight, product of:
               3.8249009 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018012092 = queryNorm
             0.3464963 = fieldWeight in 6546, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.32530996 = weight(abstract_txt:satisfaction in 6546) [ClassicSimilarity], result of:
           0.32530996 = score(doc=6546,freq=4.0), product of:
             0.4641034 = queryWeight, product of:
               4.0205555 = boost
               6.4086204 = idf(docFreq=197, maxDocs=44218)
               0.018012092 = queryNorm
             0.7009429 = fieldWeight in 6546, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.4086204 = idf(docFreq=197, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
         0.34319004 = weight(abstract_txt:engines in 6546) [ClassicSimilarity], result of:
           0.34319004 = score(doc=6546,freq=5.0), product of:
             0.5293617 = queryWeight, product of:
               5.54344 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.018012092 = queryNorm
             0.6483091 = fieldWeight in 6546, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6546)
       0.8 = coord(20/25)
   

2. Gluck, M.: Exploring the relationship between user satisfaction and relevance in information systems (1996) 0.66

   0.6564372 = sum of:
     0.6564372 = product of:
       1.094062 = sum of:
         0.016179908 = weight(abstract_txt:based in 4082) [ClassicSimilarity], result of:
           0.016179908 = score(doc=4082,freq=2.0), product of:
             0.057421245 = queryWeight, product of:
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.018012092 = queryNorm
             0.28177565 = fieldWeight in 4082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.011443095 = weight(abstract_txt:these in 4082) [ClassicSimilarity], result of:
           0.011443095 = score(doc=4082,freq=1.0), product of:
             0.05742851 = queryWeight, product of:
               1.0000632 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.018012092 = queryNorm
             0.19925809 = fieldWeight in 4082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.07471753 = weight(abstract_txt:relevance in 4082) [ClassicSimilarity], result of:
           0.07471753 = score(doc=4082,freq=7.0), product of:
             0.09161863 = queryWeight, product of:
               1.0313592 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018012092 = queryNorm
             0.8155277 = fieldWeight in 4082, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.01040089 = weight(abstract_txt:with in 4082) [ClassicSimilarity], result of:
           0.01040089 = score(doc=4082,freq=2.0), product of:
             0.047074076 = queryWeight, product of:
               1.0454998 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.018012092 = queryNorm
             0.22094731 = fieldWeight in 4082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.033173643 = weight(abstract_txt:system in 4082) [ClassicSimilarity], result of:
           0.033173643 = score(doc=4082,freq=6.0), product of:
             0.06425549 = queryWeight, product of:
               1.0578371 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.018012092 = queryNorm
             0.5162772 = fieldWeight in 4082, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.013859228 = weight(abstract_txt:also in 4082) [ClassicSimilarity], result of:
           0.013859228 = score(doc=4082,freq=1.0), product of:
             0.06525161 = queryWeight, product of:
               1.0660051 = boost
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.018012092 = queryNorm
             0.21239673 = fieldWeight in 4082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.014173043 = weight(abstract_txt:study in 4082) [ClassicSimilarity], result of:
           0.014173043 = score(doc=4082,freq=1.0), product of:
             0.06623293 = queryWeight, product of:
               1.0739911 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.018012092 = queryNorm
             0.21398787 = fieldWeight in 4082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.014913512 = weight(abstract_txt:results in 4082) [ClassicSimilarity], result of:
           0.014913512 = score(doc=4082,freq=1.0), product of:
             0.06852018 = queryWeight, product of:
               1.092378 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018012092 = queryNorm
             0.21765138 = fieldWeight in 4082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.05473862 = weight(abstract_txt:overall in 4082) [ClassicSimilarity], result of:
           0.05473862 = score(doc=4082,freq=2.0), product of:
             0.11304569 = queryWeight, product of:
               1.1456318 = boost
               5.478287 = idf(docFreq=501, maxDocs=44218)
               0.018012092 = queryNorm
             0.48421675 = fieldWeight in 4082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.478287 = idf(docFreq=501, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.034443647 = weight(abstract_txt:analysis in 4082) [ClassicSimilarity], result of:
           0.034443647 = score(doc=4082,freq=4.0), product of:
             0.07541967 = queryWeight, product of:
               1.1460564 = boost
               3.6535451 = idf(docFreq=3112, maxDocs=44218)
               0.018012092 = queryNorm
             0.45669314 = fieldWeight in 4082, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6535451 = idf(docFreq=3112, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.11341204 = weight(abstract_txt:measures in 4082) [ClassicSimilarity], result of:
           0.11341204 = score(doc=4082,freq=4.0), product of:
             0.16692369 = queryWeight, product of:
               1.7049932 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.018012092 = queryNorm
             0.67942446 = fieldWeight in 4082, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.080964796 = weight(abstract_txt:performance in 4082) [ClassicSimilarity], result of:
           0.080964796 = score(doc=4082,freq=3.0), product of:
             0.16152331 = queryWeight, product of:
               1.9366478 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.018012092 = queryNorm
             0.50125766 = fieldWeight in 4082, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.051016543 = weight(abstract_txt:significant in 4082) [ClassicSimilarity], result of:
           0.051016543 = score(doc=4082,freq=1.0), product of:
             0.17121907 = queryWeight, product of:
               1.9939264 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.018012092 = queryNorm
             0.29796064 = fieldWeight in 4082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.1152863 = weight(abstract_txt:user in 4082) [ClassicSimilarity], result of:
           0.1152863 = score(doc=4082,freq=6.0), product of:
             0.20443542 = queryWeight, product of:
               3.0812442 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.018012092 = queryNorm
             0.56392527 = fieldWeight in 4082, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
         0.4553391 = weight(abstract_txt:satisfaction in 4082) [ClassicSimilarity], result of:
           0.4553391 = score(doc=4082,freq=6.0), product of:
             0.4641034 = queryWeight, product of:
               4.0205555 = boost
               6.4086204 = idf(docFreq=197, maxDocs=44218)
               0.018012092 = queryNorm
             0.98111564 = fieldWeight in 4082, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.4086204 = idf(docFreq=197, maxDocs=44218)
               0.0625 = fieldNorm(doc=4082)
       0.6 = coord(15/25)
   

3. Su, L.T.: ¬A comprehensive and systematic model of user evaluation of Web search engines : I. Theory and background (2003) 0.63

   0.6303721 = sum of:
     0.6303721 = product of:
       1.1256645 = sum of:
         0.028240573 = weight(abstract_txt:relevance in 5164) [ClassicSimilarity], result of:
           0.028240573 = score(doc=5164,freq=1.0), product of:
             0.09161863 = queryWeight, product of:
               1.0313592 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018012092 = queryNorm
             0.3082405 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.019152813 = weight(abstract_txt:system in 5164) [ClassicSimilarity], result of:
           0.019152813 = score(doc=5164,freq=2.0), product of:
             0.06425549 = queryWeight, product of:
               1.0578371 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.018012092 = queryNorm
             0.2980728 = fieldWeight in 5164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.014913512 = weight(abstract_txt:results in 5164) [ClassicSimilarity], result of:
           0.014913512 = score(doc=5164,freq=1.0), product of:
             0.06852018 = queryWeight, product of:
               1.092378 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018012092 = queryNorm
             0.21765138 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.017221823 = weight(abstract_txt:analysis in 5164) [ClassicSimilarity], result of:
           0.017221823 = score(doc=5164,freq=1.0), product of:
             0.07541967 = queryWeight, product of:
               1.1460564 = boost
               3.6535451 = idf(docFreq=3112, maxDocs=44218)
               0.018012092 = queryNorm
             0.22834657 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6535451 = idf(docFreq=3112, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.03975374 = weight(abstract_txt:engine in 5164) [ClassicSimilarity], result of:
           0.03975374 = score(doc=5164,freq=1.0), product of:
             0.115076534 = queryWeight, product of:
               1.1558765 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.018012092 = queryNorm
             0.34545478 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.11373972 = weight(abstract_txt:undergraduates in 5164) [ClassicSimilarity], result of:
           0.11373972 = score(doc=5164,freq=1.0), product of:
             0.2319225 = queryWeight, product of:
               1.6409279 = boost
               7.84674 = idf(docFreq=46, maxDocs=44218)
               0.018012092 = queryNorm
             0.49042124 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.84674 = idf(docFreq=46, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.05670602 = weight(abstract_txt:measures in 5164) [ClassicSimilarity], result of:
           0.05670602 = score(doc=5164,freq=1.0), product of:
             0.16692369 = queryWeight, product of:
               1.7049932 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.018012092 = queryNorm
             0.33971223 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.061168626 = weight(abstract_txt:criteria in 5164) [ClassicSimilarity], result of:
           0.061168626 = score(doc=5164,freq=1.0), product of:
             0.1755703 = queryWeight, product of:
               1.7485949 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.018012092 = queryNorm
             0.34839964 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.082330994 = weight(abstract_txt:four in 5164) [ClassicSimilarity], result of:
           0.082330994 = score(doc=5164,freq=1.0), product of:
             0.2537603 = queryWeight, product of:
               2.713938 = boost
               5.191103 = idf(docFreq=668, maxDocs=44218)
               0.018012092 = queryNorm
             0.32444394 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.191103 = idf(docFreq=668, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.110439405 = weight(abstract_txt:evaluation in 5164) [ClassicSimilarity], result of:
           0.110439405 = score(doc=5164,freq=3.0), product of:
             0.22741416 = queryWeight, product of:
               2.8144104 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.018012092 = queryNorm
             0.48563117 = fieldWeight in 5164, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.066560574 = weight(abstract_txt:user in 5164) [ClassicSimilarity], result of:
           0.066560574 = score(doc=5164,freq=2.0), product of:
             0.20443542 = queryWeight, product of:
               3.0812442 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.018012092 = queryNorm
             0.32558239 = fieldWeight in 5164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.08148529 = weight(abstract_txt:search in 5164) [ClassicSimilarity], result of:
           0.08148529 = score(doc=5164,freq=2.0), product of:
             0.25201955 = queryWeight, product of:
               3.8249009 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018012092 = queryNorm
             0.3233292 = fieldWeight in 5164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.1858914 = weight(abstract_txt:satisfaction in 5164) [ClassicSimilarity], result of:
           0.1858914 = score(doc=5164,freq=1.0), product of:
             0.4641034 = queryWeight, product of:
               4.0205555 = boost
               6.4086204 = idf(docFreq=197, maxDocs=44218)
               0.018012092 = queryNorm
             0.40053877 = fieldWeight in 5164, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4086204 = idf(docFreq=197, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
         0.24805993 = weight(abstract_txt:engines in 5164) [ClassicSimilarity], result of:
           0.24805993 = score(doc=5164,freq=2.0), product of:
             0.5293617 = queryWeight, product of:
               5.54344 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.018012092 = queryNorm
             0.46860194 = fieldWeight in 5164, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0625 = fieldNorm(doc=5164)
       0.56 = coord(14/25)
   

4. Spink, A.; Jansen, B.J.; Blakely, C.; Koshman, S.: ¬A study of results overlap and uniqueness among major Web search engines (2006) 0.53

   0.5297386 = sum of:
     0.5297386 = product of:
       1.2039514 = sum of:
         0.010012708 = weight(abstract_txt:these in 993) [ClassicSimilarity], result of:
           0.010012708 = score(doc=993,freq=1.0), product of:
             0.05742851 = queryWeight, product of:
               1.0000632 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.018012092 = queryNorm
             0.17435083 = fieldWeight in 993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.006435223 = weight(abstract_txt:with in 993) [ClassicSimilarity], result of:
           0.006435223 = score(doc=993,freq=1.0), product of:
             0.047074076 = queryWeight, product of:
               1.0454998 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.018012092 = queryNorm
             0.13670418 = fieldWeight in 993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.027730403 = weight(abstract_txt:study in 993) [ClassicSimilarity], result of:
           0.027730403 = score(doc=993,freq=5.0), product of:
             0.06623293 = queryWeight, product of:
               1.0739911 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.018012092 = queryNorm
             0.41868 = fieldWeight in 993, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.03914797 = weight(abstract_txt:results in 993) [ClassicSimilarity], result of:
           0.03914797 = score(doc=993,freq=9.0), product of:
             0.06852018 = queryWeight, product of:
               1.092378 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018012092 = queryNorm
             0.5713349 = fieldWeight in 993, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.04919274 = weight(abstract_txt:engine in 993) [ClassicSimilarity], result of:
           0.04919274 = score(doc=993,freq=2.0), product of:
             0.115076534 = queryWeight, product of:
               1.1558765 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.018012092 = queryNorm
             0.42747846 = fieldWeight in 993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.040901918 = weight(abstract_txt:performance in 993) [ClassicSimilarity], result of:
           0.040901918 = score(doc=993,freq=1.0), product of:
             0.16152331 = queryWeight, product of:
               1.9366478 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.018012092 = queryNorm
             0.2532261 = fieldWeight in 993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.06312975 = weight(abstract_txt:significant in 993) [ClassicSimilarity], result of:
           0.06312975 = score(doc=993,freq=2.0), product of:
             0.17121907 = queryWeight, product of:
               1.9939264 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.018012092 = queryNorm
             0.36870748 = fieldWeight in 993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.14407925 = weight(abstract_txt:four in 993) [ClassicSimilarity], result of:
           0.14407925 = score(doc=993,freq=4.0), product of:
             0.2537603 = queryWeight, product of:
               2.713938 = boost
               5.191103 = idf(docFreq=668, maxDocs=44218)
               0.018012092 = queryNorm
             0.5677769 = fieldWeight in 993, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.191103 = idf(docFreq=668, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.041182254 = weight(abstract_txt:user in 993) [ClassicSimilarity], result of:
           0.041182254 = score(doc=993,freq=1.0), product of:
             0.20443542 = queryWeight, product of:
               3.0812442 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.018012092 = queryNorm
             0.20144382 = fieldWeight in 993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.20787235 = weight(abstract_txt:search in 993) [ClassicSimilarity], result of:
           0.20787235 = score(doc=993,freq=17.0), product of:
             0.25201955 = queryWeight, product of:
               3.8249009 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018012092 = queryNorm
             0.82482624 = fieldWeight in 993, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.5742668 = weight(abstract_txt:engines in 993) [ClassicSimilarity], result of:
           0.5742668 = score(doc=993,freq=14.0), product of:
             0.5293617 = queryWeight, product of:
               5.54344 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.018012092 = queryNorm
             1.0848287 = fieldWeight in 993, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
       0.44 = coord(11/25)
   

5. Can, F.; Nuray, R.; Sevdik, A.B.: Automatic performance evaluation of Web search engines (2004) 0.47

   0.47133267 = sum of:
     0.47133267 = product of:
       1.0712106 = sum of:
         0.020224886 = weight(abstract_txt:based in 2570) [ClassicSimilarity], result of:
           0.020224886 = score(doc=2570,freq=2.0), product of:
             0.057421245 = queryWeight, product of:
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.018012092 = queryNorm
             0.35221958 = fieldWeight in 2570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1879277 = idf(docFreq=4958, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.04992275 = weight(abstract_txt:relevance in 2570) [ClassicSimilarity], result of:
           0.04992275 = score(doc=2570,freq=2.0), product of:
             0.09161863 = queryWeight, product of:
               1.0313592 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.018012092 = queryNorm
             0.5448974 = fieldWeight in 2570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.013001113 = weight(abstract_txt:with in 2570) [ClassicSimilarity], result of:
           0.013001113 = score(doc=2570,freq=2.0), product of:
             0.047074076 = queryWeight, product of:
               1.0454998 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.018012092 = queryNorm
             0.27618414 = fieldWeight in 2570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.017716303 = weight(abstract_txt:study in 2570) [ClassicSimilarity], result of:
           0.017716303 = score(doc=2570,freq=1.0), product of:
             0.06623293 = queryWeight, product of:
               1.0739911 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.018012092 = queryNorm
             0.26748484 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.018641891 = weight(abstract_txt:results in 2570) [ClassicSimilarity], result of:
           0.018641891 = score(doc=2570,freq=1.0), product of:
             0.06852018 = queryWeight, product of:
               1.092378 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.018012092 = queryNorm
             0.27206424 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.049692176 = weight(abstract_txt:engine in 2570) [ClassicSimilarity], result of:
           0.049692176 = score(doc=2570,freq=1.0), product of:
             0.115076534 = queryWeight, product of:
               1.1558765 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.018012092 = queryNorm
             0.4318185 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.063770674 = weight(abstract_txt:significant in 2570) [ClassicSimilarity], result of:
           0.063770674 = score(doc=2570,freq=1.0), product of:
             0.17121907 = queryWeight, product of:
               1.9939264 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.018012092 = queryNorm
             0.3724508 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.11271675 = weight(abstract_txt:evaluation in 2570) [ClassicSimilarity], result of:
           0.11271675 = score(doc=2570,freq=2.0), product of:
             0.22741416 = queryWeight, product of:
               2.8144104 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.018012092 = queryNorm
             0.49564525 = fieldWeight in 2570, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.05883179 = weight(abstract_txt:user in 2570) [ClassicSimilarity], result of:
           0.05883179 = score(doc=2570,freq=1.0), product of:
             0.20443542 = queryWeight, product of:
               3.0812442 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.018012092 = queryNorm
             0.2877769 = fieldWeight in 2570, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.17642082 = weight(abstract_txt:search in 2570) [ClassicSimilarity], result of:
           0.17642082 = score(doc=2570,freq=6.0), product of:
             0.25201955 = queryWeight, product of:
               3.8249009 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.018012092 = queryNorm
             0.7000283 = fieldWeight in 2570, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
         0.4902715 = weight(abstract_txt:engines in 2570) [ClassicSimilarity], result of:
           0.4902715 = score(doc=2570,freq=5.0), product of:
             0.5293617 = queryWeight, product of:
               5.54344 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.018012092 = queryNorm
             0.9261559 = fieldWeight in 2570, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.078125 = fieldNorm(doc=2570)
       0.44 = coord(11/25)