Document (#27119)

Author
Su, L.T.
Title
¬A comprehensive and systematic model of user evaluation of Web search engines : Il. An evaluation by undergraduates
Source
Journal of the American Society for Information Science and technology. 54(2003) no.13, S.1193-1222
Year
2003
Abstract
This paper presents an application of the model described in Part I to the evaluation of Web search engines by undergraduates. The study observed how 36 undergraduate used four major search engines to find information for their own individual problems and how they evaluated these engines based an actual interaction with the search engines. User evaluation was based an 16 performance measures representing five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Non-performance (user-related) measures were also applied. Each participant searched his/ her own topic an all four engines and provided satisfaction ratings for system features and interaction and reasons for satisfaction. Each also made relevance judgements of retrieved items in relation to his/her own information need and participated in post-search Interviews to provide reactions to the search results and overall performance. The study found significant differences in precision PR1 relative recall, user satisfaction with output display, time saving, value of search results, and overall performance among the four engines and also significant engine by discipline interactions an all these measures. In addition, the study found significant differences in user satisfaction with response time among four engines, and significant engine by discipline interaction in user satisfaction with search interface. None of the four search engines dominated in every aspect of the multidimensional evaluation. Content analysis of verbal data identified a number of user criteria and users evaluative comments based an these criteria. Results from both quantitative analysis and content analysis provide insight for system design and development, and useful feedback an strengths and weaknesses of search engines for system improvement
Footnote
Teil I im selben Heft
Theme
Suchmaschinen
Benutzerstudien

Similar documents (content)

  1. Su, L.T.; Chen, H.L.: Evaluation of Web search engines by undergraduate students (1999) 1.25
    1.2466673 = sum of:
      1.2466673 = product of:
        1.5583341 = sum of:
          0.0143457595 = weight(abstract_txt:based in 1547) [ClassicSimilarity], result of:
            0.0143457595 = score(doc=1547,freq=2.0), product of:
              0.05792902 = queryWeight, product of:
                3.2020218 = idf(docFreq=4782, maxDocs=43254)
                0.018091388 = queryNorm
              0.24764374 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2020218 = idf(docFreq=4782, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.014424817 = weight(abstract_txt:these in 1547) [ClassicSimilarity], result of:
            0.014424817 = score(doc=1547,freq=2.0), product of:
              0.05814165 = queryWeight, product of:
                1.0018336 = boost
                3.2078931 = idf(docFreq=4754, maxDocs=43254)
                0.018091388 = queryNorm
              0.24809782 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2078931 = idf(docFreq=4754, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.042832796 = weight(abstract_txt:relevance in 1547) [ClassicSimilarity], result of:
            0.042832796 = score(doc=1547,freq=3.0), product of:
              0.091665074 = queryWeight, product of:
                1.0270894 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.018091388 = queryNorm
              0.46727496 = fieldWeight in 1547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.025426278 = weight(abstract_txt:differences in 1547) [ClassicSimilarity], result of:
            0.025426278 = score(doc=1547,freq=1.0), product of:
              0.093378864 = queryWeight, product of:
                1.0366462 = boost
                4.979046 = idf(docFreq=808, maxDocs=43254)
                0.018091388 = queryNorm
              0.27229157 = fieldWeight in 1547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.979046 = idf(docFreq=808, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.01664487 = weight(abstract_txt:system in 1547) [ClassicSimilarity], result of:
            0.01664487 = score(doc=1547,freq=2.0), product of:
              0.06396378 = queryWeight, product of:
                1.0507973 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018091388 = queryNorm
              0.2602234 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.017979305 = weight(abstract_txt:study in 1547) [ClassicSimilarity], result of:
            0.017979305 = score(doc=1547,freq=2.0), product of:
              0.06733835 = queryWeight, product of:
                1.0781598 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.018091388 = queryNorm
              0.2669995 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.018746382 = weight(abstract_txt:results in 1547) [ClassicSimilarity], result of:
            0.018746382 = score(doc=1547,freq=2.0), product of:
              0.06924027 = queryWeight, product of:
                1.0932797 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.018091388 = queryNorm
              0.2707439 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.03413629 = weight(abstract_txt:overall in 1547) [ClassicSimilarity], result of:
            0.03413629 = score(doc=1547,freq=1.0), product of:
              0.11364167 = queryWeight, product of:
                1.1436025 = boost
                5.49276 = idf(docFreq=483, maxDocs=43254)
                0.018091388 = queryNorm
              0.30038533 = fieldWeight in 1547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.49276 = idf(docFreq=483, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.048884705 = weight(abstract_txt:engine in 1547) [ClassicSimilarity], result of:
            0.048884705 = score(doc=1547,freq=2.0), product of:
              0.11459493 = queryWeight, product of:
                1.1483889 = boost
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.018091388 = queryNorm
              0.426587 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.04496043 = weight(abstract_txt:interaction in 1547) [ClassicSimilarity], result of:
            0.04496043 = score(doc=1547,freq=1.0), product of:
              0.15630665 = queryWeight, product of:
                1.6426333 = boost
                5.259748 = idf(docFreq=610, maxDocs=43254)
                0.018091388 = queryNorm
              0.28764248 = fieldWeight in 1547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.259748 = idf(docFreq=610, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.085440576 = weight(abstract_txt:measures in 1547) [ClassicSimilarity], result of:
            0.085440576 = score(doc=1547,freq=3.0), product of:
              0.1662747 = queryWeight, product of:
                1.6942012 = boost
                5.4248695 = idf(docFreq=517, maxDocs=43254)
                0.018091388 = queryNorm
              0.51385194 = fieldWeight in 1547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.4248695 = idf(docFreq=517, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.05385076 = weight(abstract_txt:criteria in 1547) [ClassicSimilarity], result of:
            0.05385076 = score(doc=1547,freq=1.0), product of:
              0.17628622 = queryWeight, product of:
                1.7444603 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.018091388 = queryNorm
              0.30547345 = fieldWeight in 1547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.058120634 = weight(abstract_txt:performance in 1547) [ClassicSimilarity], result of:
            0.058120634 = score(doc=1547,freq=2.0), product of:
              0.16203679 = queryWeight, product of:
                1.9312037 = boost
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.018091388 = queryNorm
              0.35868788 = fieldWeight in 1547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.045200605 = weight(abstract_txt:significant in 1547) [ClassicSimilarity], result of:
            0.045200605 = score(doc=1547,freq=1.0), product of:
              0.17264986 = queryWeight, product of:
                1.9934456 = boost
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.018091388 = queryNorm
              0.26180506 = fieldWeight in 1547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.14554672 = weight(abstract_txt:four in 1547) [ClassicSimilarity], result of:
            0.14554672 = score(doc=1547,freq=4.0), product of:
              0.2554789 = queryWeight, product of:
                2.7111516 = boost
                5.2087 = idf(docFreq=642, maxDocs=43254)
                0.018091388 = queryNorm
              0.56970155 = fieldWeight in 1547, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.2087 = idf(docFreq=642, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.096900925 = weight(abstract_txt:evaluation in 1547) [ClassicSimilarity], result of:
            0.096900925 = score(doc=1547,freq=3.0), product of:
              0.22783059 = queryWeight, product of:
                2.8046129 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018091388 = queryNorm
              0.4253201 = fieldWeight in 1547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.041036002 = weight(abstract_txt:user in 1547) [ClassicSimilarity], result of:
            0.041036002 = score(doc=1547,freq=1.0), product of:
              0.20394987 = queryWeight, product of:
                3.0640652 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018091388 = queryNorm
              0.20120633 = fieldWeight in 1547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.08695605 = weight(abstract_txt:search in 1547) [ClassicSimilarity], result of:
            0.08695605 = score(doc=1547,freq=3.0), product of:
              0.25130984 = queryWeight, product of:
                3.8027346 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.018091388 = queryNorm
              0.3460113 = fieldWeight in 1547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.3250552 = weight(abstract_txt:satisfaction in 1547) [ClassicSimilarity], result of:
            0.3250552 = score(doc=1547,freq=4.0), product of:
              0.46385822 = queryWeight, product of:
                4.001838 = boost
                6.4069867 = idf(docFreq=193, maxDocs=43254)
                0.018091388 = queryNorm
              0.7007642 = fieldWeight in 1547, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.4069867 = idf(docFreq=193, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
          0.3418449 = weight(abstract_txt:engines in 1547) [ClassicSimilarity], result of:
            0.3418449 = score(doc=1547,freq=5.0), product of:
              0.5279743 = queryWeight, product of:
                5.5118537 = boost
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.018091388 = queryNorm
              0.647465 = fieldWeight in 1547, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1547)
        0.8 = coord(20/25)
    
  2. Gluck, M.: Exploring the relationship between user satisfaction and relevance in information systems (1996) 0.66
    0.65754384 = sum of:
      0.65754384 = product of:
        1.0959064 = sum of:
          0.016395153 = weight(abstract_txt:based in 5151) [ClassicSimilarity], result of:
            0.016395153 = score(doc=5151,freq=2.0), product of:
              0.05792902 = queryWeight, product of:
                3.2020218 = idf(docFreq=4782, maxDocs=43254)
                0.018091388 = queryNorm
              0.28302142 = fieldWeight in 5151, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2020218 = idf(docFreq=4782, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.011657013 = weight(abstract_txt:these in 5151) [ClassicSimilarity], result of:
            0.011657013 = score(doc=5151,freq=1.0), product of:
              0.05814165 = queryWeight, product of:
                1.0018336 = boost
                3.2078931 = idf(docFreq=4754, maxDocs=43254)
                0.018091388 = queryNorm
              0.20049332 = fieldWeight in 5151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2078931 = idf(docFreq=4754, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.074775055 = weight(abstract_txt:relevance in 5151) [ClassicSimilarity], result of:
            0.074775055 = score(doc=5151,freq=7.0), product of:
              0.091665074 = queryWeight, product of:
                1.0270894 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.018091388 = queryNorm
              0.815742 = fieldWeight in 5151, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.010537596 = weight(abstract_txt:with in 5151) [ClassicSimilarity], result of:
            0.010537596 = score(doc=5151,freq=2.0), product of:
              0.04748537 = queryWeight, product of:
                1.045445 = boost
                2.5106533 = idf(docFreq=9548, maxDocs=43254)
                0.018091388 = queryNorm
              0.22191249 = fieldWeight in 5151, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5106533 = idf(docFreq=9548, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.0329483 = weight(abstract_txt:system in 5151) [ClassicSimilarity], result of:
            0.0329483 = score(doc=5151,freq=6.0), product of:
              0.06396378 = queryWeight, product of:
                1.0507973 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018091388 = queryNorm
              0.5151087 = fieldWeight in 5151, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.014110173 = weight(abstract_txt:also in 5151) [ClassicSimilarity], result of:
            0.014110173 = score(doc=5151,freq=1.0), product of:
              0.06603651 = queryWeight, product of:
                1.067687 = boost
                3.418757 = idf(docFreq=3850, maxDocs=43254)
                0.018091388 = queryNorm
              0.21367231 = fieldWeight in 5151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.418757 = idf(docFreq=3850, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.014529474 = weight(abstract_txt:study in 5151) [ClassicSimilarity], result of:
            0.014529474 = score(doc=5151,freq=1.0), product of:
              0.06733835 = queryWeight, product of:
                1.0781598 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.018091388 = queryNorm
              0.2157682 = fieldWeight in 5151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.015149363 = weight(abstract_txt:results in 5151) [ClassicSimilarity], result of:
            0.015149363 = score(doc=5151,freq=1.0), product of:
              0.06924027 = queryWeight, product of:
                1.0932797 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.018091388 = queryNorm
              0.2187941 = fieldWeight in 5151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.055172574 = weight(abstract_txt:overall in 5151) [ClassicSimilarity], result of:
            0.055172574 = score(doc=5151,freq=2.0), product of:
              0.11364167 = queryWeight, product of:
                1.1436025 = boost
                5.49276 = idf(docFreq=483, maxDocs=43254)
                0.018091388 = queryNorm
              0.48549598 = fieldWeight in 5151, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.49276 = idf(docFreq=483, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.035010166 = weight(abstract_txt:analysis in 5151) [ClassicSimilarity], result of:
            0.035010166 = score(doc=5151,freq=4.0), product of:
              0.07624394 = queryWeight, product of:
                1.1472408 = boost
                3.67349 = idf(docFreq=2984, maxDocs=43254)
                0.018091388 = queryNorm
              0.45918626 = fieldWeight in 5151, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.67349 = idf(docFreq=2984, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.11275232 = weight(abstract_txt:measures in 5151) [ClassicSimilarity], result of:
            0.11275232 = score(doc=5151,freq=4.0), product of:
              0.1662747 = queryWeight, product of:
                1.6942012 = boost
                5.4248695 = idf(docFreq=517, maxDocs=43254)
                0.018091388 = queryNorm
              0.6781087 = fieldWeight in 5151, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.4248695 = idf(docFreq=517, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.08135194 = weight(abstract_txt:performance in 5151) [ClassicSimilarity], result of:
            0.08135194 = score(doc=5151,freq=3.0), product of:
              0.16203679 = queryWeight, product of:
                1.9312037 = boost
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.018091388 = queryNorm
              0.50205845 = fieldWeight in 5151, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.051657837 = weight(abstract_txt:significant in 5151) [ClassicSimilarity], result of:
            0.051657837 = score(doc=5151,freq=1.0), product of:
              0.17264986 = queryWeight, product of:
                1.9934456 = boost
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.018091388 = queryNorm
              0.29920578 = fieldWeight in 5151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.11487688 = weight(abstract_txt:user in 5151) [ClassicSimilarity], result of:
            0.11487688 = score(doc=5151,freq=6.0), product of:
              0.20394987 = queryWeight, product of:
                3.0640652 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018091388 = queryNorm
              0.5632604 = fieldWeight in 5151, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
          0.45498255 = weight(abstract_txt:satisfaction in 5151) [ClassicSimilarity], result of:
            0.45498255 = score(doc=5151,freq=6.0), product of:
              0.46385822 = queryWeight, product of:
                4.001838 = boost
                6.4069867 = idf(docFreq=193, maxDocs=43254)
                0.018091388 = queryNorm
              0.98086554 = fieldWeight in 5151, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                6.4069867 = idf(docFreq=193, maxDocs=43254)
                0.0625 = fieldNorm(doc=5151)
        0.6 = coord(15/25)
    
  3. Su, L.T.: ¬A comprehensive and systematic model of user evaluation of Web search engines : I. Theory and background (2003) 0.63
    0.6296411 = sum of:
      0.6296411 = product of:
        1.1243591 = sum of:
          0.028262313 = weight(abstract_txt:relevance in 165) [ClassicSimilarity], result of:
            0.028262313 = score(doc=165,freq=1.0), product of:
              0.091665074 = queryWeight, product of:
                1.0270894 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.018091388 = queryNorm
              0.3083215 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.019022709 = weight(abstract_txt:system in 165) [ClassicSimilarity], result of:
            0.019022709 = score(doc=165,freq=2.0), product of:
              0.06396378 = queryWeight, product of:
                1.0507973 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018091388 = queryNorm
              0.29739815 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.015149363 = weight(abstract_txt:results in 165) [ClassicSimilarity], result of:
            0.015149363 = score(doc=165,freq=1.0), product of:
              0.06924027 = queryWeight, product of:
                1.0932797 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.018091388 = queryNorm
              0.2187941 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.017505083 = weight(abstract_txt:analysis in 165) [ClassicSimilarity], result of:
            0.017505083 = score(doc=165,freq=1.0), product of:
              0.07624394 = queryWeight, product of:
                1.1472408 = boost
                3.67349 = idf(docFreq=2984, maxDocs=43254)
                0.018091388 = queryNorm
              0.22959313 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.67349 = idf(docFreq=2984, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.039504807 = weight(abstract_txt:engine in 165) [ClassicSimilarity], result of:
            0.039504807 = score(doc=165,freq=1.0), product of:
              0.11459493 = queryWeight, product of:
                1.1483889 = boost
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.018091388 = queryNorm
              0.34473434 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.11278179 = weight(abstract_txt:undergraduates in 165) [ClassicSimilarity], result of:
            0.11278179 = score(doc=165,freq=1.0), product of:
              0.23061706 = queryWeight, product of:
                1.6291159 = boost
                7.8246975 = idf(docFreq=46, maxDocs=43254)
                0.018091388 = queryNorm
              0.4890436 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.8246975 = idf(docFreq=46, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.05637616 = weight(abstract_txt:measures in 165) [ClassicSimilarity], result of:
            0.05637616 = score(doc=165,freq=1.0), product of:
              0.1662747 = queryWeight, product of:
                1.6942012 = boost
                5.4248695 = idf(docFreq=517, maxDocs=43254)
                0.018091388 = queryNorm
              0.33905435 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.4248695 = idf(docFreq=517, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.061543725 = weight(abstract_txt:criteria in 165) [ClassicSimilarity], result of:
            0.061543725 = score(doc=165,freq=1.0), product of:
              0.17628622 = queryWeight, product of:
                1.7444603 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.018091388 = queryNorm
              0.3491125 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.08316956 = weight(abstract_txt:four in 165) [ClassicSimilarity], result of:
            0.08316956 = score(doc=165,freq=1.0), product of:
              0.2554789 = queryWeight, product of:
                2.7111516 = boost
                5.2087 = idf(docFreq=642, maxDocs=43254)
                0.018091388 = queryNorm
              0.32554376 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.2087 = idf(docFreq=642, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.11074392 = weight(abstract_txt:evaluation in 165) [ClassicSimilarity], result of:
            0.11074392 = score(doc=165,freq=3.0), product of:
              0.22783059 = queryWeight, product of:
                2.8046129 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018091388 = queryNorm
              0.4860801 = fieldWeight in 165, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.0663242 = weight(abstract_txt:user in 165) [ClassicSimilarity], result of:
            0.0663242 = score(doc=165,freq=2.0), product of:
              0.20394987 = queryWeight, product of:
                3.0640652 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018091388 = queryNorm
              0.32519853 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.081142075 = weight(abstract_txt:search in 165) [ClassicSimilarity], result of:
            0.081142075 = score(doc=165,freq=2.0), product of:
              0.25130984 = queryWeight, product of:
                3.8027346 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.018091388 = queryNorm
              0.32287663 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.18574584 = weight(abstract_txt:satisfaction in 165) [ClassicSimilarity], result of:
            0.18574584 = score(doc=165,freq=1.0), product of:
              0.46385822 = queryWeight, product of:
                4.001838 = boost
                6.4069867 = idf(docFreq=193, maxDocs=43254)
                0.018091388 = queryNorm
              0.40043667 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.4069867 = idf(docFreq=193, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
          0.24708763 = weight(abstract_txt:engines in 165) [ClassicSimilarity], result of:
            0.24708763 = score(doc=165,freq=2.0), product of:
              0.5279743 = queryWeight, product of:
                5.5118537 = boost
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.018091388 = queryNorm
              0.46799177 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.0625 = fieldNorm(doc=165)
        0.56 = coord(14/25)
    
  4. Spink, A.; Jansen, B.J.; Blakely, C.; Koshman, S.: ¬A study of results overlap and uniqueness among major Web search engines (2006) 0.53
    0.52994287 = sum of:
      0.52994287 = product of:
        1.2044156 = sum of:
          0.010199886 = weight(abstract_txt:these in 2994) [ClassicSimilarity], result of:
            0.010199886 = score(doc=2994,freq=1.0), product of:
              0.05814165 = queryWeight, product of:
                1.0018336 = boost
                3.2078931 = idf(docFreq=4754, maxDocs=43254)
                0.018091388 = queryNorm
              0.17543165 = fieldWeight in 2994, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2078931 = idf(docFreq=4754, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.0065198056 = weight(abstract_txt:with in 2994) [ClassicSimilarity], result of:
            0.0065198056 = score(doc=2994,freq=1.0), product of:
              0.04748537 = queryWeight, product of:
                1.045445 = boost
                2.5106533 = idf(docFreq=9548, maxDocs=43254)
                0.018091388 = queryNorm
              0.13730136 = fieldWeight in 2994, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.5106533 = idf(docFreq=9548, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.028427782 = weight(abstract_txt:study in 2994) [ClassicSimilarity], result of:
            0.028427782 = score(doc=2994,freq=5.0), product of:
              0.06733835 = queryWeight, product of:
                1.0781598 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.018091388 = queryNorm
              0.42216334 = fieldWeight in 2994, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.039767075 = weight(abstract_txt:results in 2994) [ClassicSimilarity], result of:
            0.039767075 = score(doc=2994,freq=9.0), product of:
              0.06924027 = queryWeight, product of:
                1.0932797 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.018091388 = queryNorm
              0.5743345 = fieldWeight in 2994, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.048884705 = weight(abstract_txt:engine in 2994) [ClassicSimilarity], result of:
            0.048884705 = score(doc=2994,freq=2.0), product of:
              0.11459493 = queryWeight, product of:
                1.1483889 = boost
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.018091388 = queryNorm
              0.426587 = fieldWeight in 2994, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.041097496 = weight(abstract_txt:performance in 2994) [ClassicSimilarity], result of:
            0.041097496 = score(doc=2994,freq=1.0), product of:
              0.16203679 = queryWeight, product of:
                1.9312037 = boost
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.018091388 = queryNorm
              0.25363064 = fieldWeight in 2994, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.06392331 = weight(abstract_txt:significant in 2994) [ClassicSimilarity], result of:
            0.06392331 = score(doc=2994,freq=2.0), product of:
              0.17264986 = queryWeight, product of:
                1.9934456 = boost
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.018091388 = queryNorm
              0.37024826 = fieldWeight in 2994, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.14554672 = weight(abstract_txt:four in 2994) [ClassicSimilarity], result of:
            0.14554672 = score(doc=2994,freq=4.0), product of:
              0.2554789 = queryWeight, product of:
                2.7111516 = boost
                5.2087 = idf(docFreq=642, maxDocs=43254)
                0.018091388 = queryNorm
              0.56970155 = fieldWeight in 2994, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.2087 = idf(docFreq=642, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.041036002 = weight(abstract_txt:user in 2994) [ClassicSimilarity], result of:
            0.041036002 = score(doc=2994,freq=1.0), product of:
              0.20394987 = queryWeight, product of:
                3.0640652 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018091388 = queryNorm
              0.20120633 = fieldWeight in 2994, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.2069968 = weight(abstract_txt:search in 2994) [ClassicSimilarity], result of:
            0.2069968 = score(doc=2994,freq=17.0), product of:
              0.25130984 = queryWeight, product of:
                3.8027346 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.018091388 = queryNorm
              0.8236717 = fieldWeight in 2994, product of:
                4.1231055 = tf(freq=17.0), with freq of:
                  17.0 = termFreq=17.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
          0.57201594 = weight(abstract_txt:engines in 2994) [ClassicSimilarity], result of:
            0.57201594 = score(doc=2994,freq=14.0), product of:
              0.5279743 = queryWeight, product of:
                5.5118537 = boost
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.018091388 = queryNorm
              1.0834162 = fieldWeight in 2994, product of:
                3.7416575 = tf(freq=14.0), with freq of:
                  14.0 = termFreq=14.0
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.0546875 = fieldNorm(doc=2994)
        0.44 = coord(11/25)
    
  5. Can, F.; Nuray, R.; Sevdik, A.B.: Automatic performance evaluation of Web search engines (2004) 0.47
    0.47095707 = sum of:
      0.47095707 = product of:
        1.070357 = sum of:
          0.020493943 = weight(abstract_txt:based in 4571) [ClassicSimilarity], result of:
            0.020493943 = score(doc=4571,freq=2.0), product of:
              0.05792902 = queryWeight, product of:
                3.2020218 = idf(docFreq=4782, maxDocs=43254)
                0.018091388 = queryNorm
              0.35377678 = fieldWeight in 4571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2020218 = idf(docFreq=4782, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.049961183 = weight(abstract_txt:relevance in 4571) [ClassicSimilarity], result of:
            0.049961183 = score(doc=4571,freq=2.0), product of:
              0.091665074 = queryWeight, product of:
                1.0270894 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.018091388 = queryNorm
              0.54504055 = fieldWeight in 4571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.013171995 = weight(abstract_txt:with in 4571) [ClassicSimilarity], result of:
            0.013171995 = score(doc=4571,freq=2.0), product of:
              0.04748537 = queryWeight, product of:
                1.045445 = boost
                2.5106533 = idf(docFreq=9548, maxDocs=43254)
                0.018091388 = queryNorm
              0.2773906 = fieldWeight in 4571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5106533 = idf(docFreq=9548, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.018161843 = weight(abstract_txt:study in 4571) [ClassicSimilarity], result of:
            0.018161843 = score(doc=4571,freq=1.0), product of:
              0.06733835 = queryWeight, product of:
                1.0781598 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.018091388 = queryNorm
              0.26971024 = fieldWeight in 4571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.018936705 = weight(abstract_txt:results in 4571) [ClassicSimilarity], result of:
            0.018936705 = score(doc=4571,freq=1.0), product of:
              0.06924027 = queryWeight, product of:
                1.0932797 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.018091388 = queryNorm
              0.27349263 = fieldWeight in 4571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.049381007 = weight(abstract_txt:engine in 4571) [ClassicSimilarity], result of:
            0.049381007 = score(doc=4571,freq=1.0), product of:
              0.11459493 = queryWeight, product of:
                1.1483889 = boost
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.018091388 = queryNorm
              0.43091792 = fieldWeight in 4571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5157495 = idf(docFreq=472, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.0645723 = weight(abstract_txt:significant in 4571) [ClassicSimilarity], result of:
            0.0645723 = score(doc=4571,freq=1.0), product of:
              0.17264986 = queryWeight, product of:
                1.9934456 = boost
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.018091388 = queryNorm
              0.37400723 = fieldWeight in 4571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.7872925 = idf(docFreq=979, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.113027535 = weight(abstract_txt:evaluation in 4571) [ClassicSimilarity], result of:
            0.113027535 = score(doc=4571,freq=2.0), product of:
              0.22783059 = queryWeight, product of:
                2.8046129 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018091388 = queryNorm
              0.49610344 = fieldWeight in 4571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.058622863 = weight(abstract_txt:user in 4571) [ClassicSimilarity], result of:
            0.058622863 = score(doc=4571,freq=1.0), product of:
              0.20394987 = queryWeight, product of:
                3.0640652 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018091388 = queryNorm
              0.28743762 = fieldWeight in 4571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.17567778 = weight(abstract_txt:search in 4571) [ClassicSimilarity], result of:
            0.17567778 = score(doc=4571,freq=6.0), product of:
              0.25130984 = queryWeight, product of:
                3.8027346 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.018091388 = queryNorm
              0.6990485 = fieldWeight in 4571, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
          0.4883498 = weight(abstract_txt:engines in 4571) [ClassicSimilarity], result of:
            0.4883498 = score(doc=4571,freq=5.0), product of:
              0.5279743 = queryWeight, product of:
                5.5118537 = boost
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.018091388 = queryNorm
              0.92494994 = fieldWeight in 4571, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.2947226 = idf(docFreq=589, maxDocs=43254)
                0.078125 = fieldNorm(doc=4571)
        0.44 = coord(11/25)