Document (#27119)

Author
Su, L.T.
Title
¬A comprehensive and systematic model of user evaluation of Web search engines : Il. An evaluation by undergraduates
Source
Journal of the American Society for Information Science and technology. 54(2003) no.13, S.1193-1222
Year
2003
Abstract
This paper presents an application of the model described in Part I to the evaluation of Web search engines by undergraduates. The study observed how 36 undergraduate used four major search engines to find information for their own individual problems and how they evaluated these engines based an actual interaction with the search engines. User evaluation was based an 16 performance measures representing five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Non-performance (user-related) measures were also applied. Each participant searched his/ her own topic an all four engines and provided satisfaction ratings for system features and interaction and reasons for satisfaction. Each also made relevance judgements of retrieved items in relation to his/her own information need and participated in post-search Interviews to provide reactions to the search results and overall performance. The study found significant differences in precision PR1 relative recall, user satisfaction with output display, time saving, value of search results, and overall performance among the four engines and also significant engine by discipline interactions an all these measures. In addition, the study found significant differences in user satisfaction with response time among four engines, and significant engine by discipline interaction in user satisfaction with search interface. None of the four search engines dominated in every aspect of the multidimensional evaluation. Content analysis of verbal data identified a number of user criteria and users evaluative comments based an these criteria. Results from both quantitative analysis and content analysis provide insight for system design and development, and useful feedback an strengths and weaknesses of search engines for system improvement
Footnote
Teil I im selben Heft
Theme
Suchmaschinen
Benutzerstudien

Similar documents (content)

  1. Su, L.T.; Chen, H.L.: Evaluation of Web search engines by undergraduate students (1999) 1.25
    1.2478844 = sum of:
      1.2478844 = product of:
        1.5598555 = sum of:
          0.014410334 = weight(abstract_txt:based in 547) [ClassicSimilarity], result of:
            0.014410334 = score(doc=547,freq=2.0), product of:
              0.058069106 = queryWeight, product of:
                3.2086759 = idf(docFreq=4678, maxDocs=42596)
                0.01809753 = queryNorm
              0.24815835 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2086759 = idf(docFreq=4678, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.0145614315 = weight(abstract_txt:these in 547) [ClassicSimilarity], result of:
            0.0145614315 = score(doc=547,freq=2.0), product of:
              0.058474317 = queryWeight, product of:
                1.0034829 = boost
                3.2198517 = idf(docFreq=4626, maxDocs=42596)
                0.01809753 = queryNorm
              0.24902269 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2198517 = idf(docFreq=4626, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.04273092 = weight(abstract_txt:relevance in 547) [ClassicSimilarity], result of:
            0.04273092 = score(doc=547,freq=3.0), product of:
              0.09146672 = queryWeight, product of:
                1.0247391 = boost
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.01809753 = queryNorm
              0.46717453 = fieldWeight in 547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.025492 = weight(abstract_txt:differences in 547) [ClassicSimilarity], result of:
            0.025492 = score(doc=547,freq=1.0), product of:
              0.09348558 = queryWeight, product of:
                1.0359865 = boost
                4.9862175 = idf(docFreq=790, maxDocs=42596)
                0.01809753 = queryNorm
              0.27268377 = fieldWeight in 547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.9862175 = idf(docFreq=790, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.016587026 = weight(abstract_txt:system in 547) [ClassicSimilarity], result of:
            0.016587026 = score(doc=547,freq=2.0), product of:
              0.06377857 = queryWeight, product of:
                1.0480086 = boost
                3.3627198 = idf(docFreq=4010, maxDocs=42596)
                0.01809753 = queryNorm
              0.26007208 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.3627198 = idf(docFreq=4010, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.018278243 = weight(abstract_txt:study in 547) [ClassicSimilarity], result of:
            0.018278243 = score(doc=547,freq=2.0), product of:
              0.0680433 = queryWeight, product of:
                1.0824805 = boost
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.01809753 = queryNorm
              0.26862663 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.018815456 = weight(abstract_txt:results in 547) [ClassicSimilarity], result of:
            0.018815456 = score(doc=547,freq=2.0), product of:
              0.06937009 = queryWeight, product of:
                1.0929834 = boost
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.01809753 = queryNorm
              0.27123296 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.034219828 = weight(abstract_txt:overall in 547) [ClassicSimilarity], result of:
            0.034219828 = score(doc=547,freq=1.0), product of:
              0.11376113 = queryWeight, product of:
                1.1428224 = boost
                5.50042 = idf(docFreq=472, maxDocs=42596)
                0.01809753 = queryNorm
              0.30080423 = fieldWeight in 547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.50042 = idf(docFreq=472, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.04878875 = weight(abstract_txt:engine in 547) [ClassicSimilarity], result of:
            0.04878875 = score(doc=547,freq=2.0), product of:
              0.11437869 = queryWeight, product of:
                1.1459202 = boost
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.01809753 = queryNorm
              0.42655453 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.04521317 = weight(abstract_txt:interaction in 547) [ClassicSimilarity], result of:
            0.04521317 = score(doc=547,freq=1.0), product of:
              0.15680109 = queryWeight, product of:
                1.6432437 = boost
                5.2726364 = idf(docFreq=593, maxDocs=42596)
                0.01809753 = queryNorm
              0.2883473 = fieldWeight in 547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.2726364 = idf(docFreq=593, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.08576903 = weight(abstract_txt:measures in 547) [ClassicSimilarity], result of:
            0.08576903 = score(doc=547,freq=3.0), product of:
              0.16660409 = queryWeight, product of:
                1.6938316 = boost
                5.434957 = idf(docFreq=504, maxDocs=42596)
                0.01809753 = queryNorm
              0.51480746 = fieldWeight in 547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.434957 = idf(docFreq=504, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.05371026 = weight(abstract_txt:criteria in 547) [ClassicSimilarity], result of:
            0.05371026 = score(doc=547,freq=1.0), product of:
              0.17587762 = queryWeight, product of:
                1.7403344 = boost
                5.5841694 = idf(docFreq=434, maxDocs=42596)
                0.01809753 = queryNorm
              0.30538428 = fieldWeight in 547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5841694 = idf(docFreq=434, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.05814359 = weight(abstract_txt:performance in 547) [ClassicSimilarity], result of:
            0.05814359 = score(doc=547,freq=2.0), product of:
              0.16198567 = queryWeight, product of:
                1.9285686 = boost
                4.6411138 = idf(docFreq=1116, maxDocs=42596)
                0.01809753 = queryNorm
              0.3589428 = fieldWeight in 547, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6411138 = idf(docFreq=1116, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.04553938 = weight(abstract_txt:significant in 547) [ClassicSimilarity], result of:
            0.04553938 = score(doc=547,freq=1.0), product of:
              0.17341104 = queryWeight, product of:
                1.9954238 = boost
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.01809753 = queryNorm
              0.26260945 = fieldWeight in 547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.14706674 = weight(abstract_txt:four in 547) [ClassicSimilarity], result of:
            0.14706674 = score(doc=547,freq=4.0), product of:
              0.25710568 = queryWeight, product of:
                2.716487 = boost
                5.229796 = idf(docFreq=619, maxDocs=42596)
                0.01809753 = queryNorm
              0.5720089 = fieldWeight in 547, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.229796 = idf(docFreq=619, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.09682916 = weight(abstract_txt:evaluation in 547) [ClassicSimilarity], result of:
            0.09682916 = score(doc=547,freq=3.0), product of:
              0.2275863 = queryWeight, product of:
                2.7997255 = boost
                4.4917064 = idf(docFreq=1296, maxDocs=42596)
                0.01809753 = queryNorm
              0.4254613 = fieldWeight in 547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.4917064 = idf(docFreq=1296, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.040951658 = weight(abstract_txt:user in 547) [ClassicSimilarity], result of:
            0.040951658 = score(doc=547,freq=1.0), product of:
              0.20355245 = queryWeight, product of:
                3.0573838 = boost
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.01809753 = queryNorm
              0.2011848 = fieldWeight in 547, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.08674933 = weight(abstract_txt:search in 547) [ClassicSimilarity], result of:
            0.08674933 = score(doc=547,freq=3.0), product of:
              0.25076622 = queryWeight, product of:
                3.7940347 = boost
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.01809753 = queryNorm
              0.34593707 = fieldWeight in 547, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.32373726 = weight(abstract_txt:satisfaction in 547) [ClassicSimilarity], result of:
            0.32373726 = score(doc=547,freq=4.0), product of:
              0.46233582 = queryWeight, product of:
                3.990444 = boost
                6.4020205 = idf(docFreq=191, maxDocs=42596)
                0.01809753 = queryNorm
              0.700221 = fieldWeight in 547, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.4020205 = idf(docFreq=191, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
          0.34226182 = weight(abstract_txt:engines in 547) [ClassicSimilarity], result of:
            0.34226182 = score(doc=547,freq=5.0), product of:
              0.52809775 = queryWeight, product of:
                5.5058446 = boost
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.01809753 = queryNorm
              0.6481032 = fieldWeight in 547, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.0546875 = fieldNorm(doc=547)
        0.8 = coord(20/25)
    
  2. Gluck, M.: Exploring the relationship between user satisfaction and relevance in information systems (1996) 0.66
    0.6573734 = sum of:
      0.6573734 = product of:
        1.0956223 = sum of:
          0.016468953 = weight(abstract_txt:based in 4151) [ClassicSimilarity], result of:
            0.016468953 = score(doc=4151,freq=2.0), product of:
              0.058069106 = queryWeight, product of:
                3.2086759 = idf(docFreq=4678, maxDocs=42596)
                0.01809753 = queryNorm
              0.28360954 = fieldWeight in 4151, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2086759 = idf(docFreq=4678, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.011767414 = weight(abstract_txt:these in 4151) [ClassicSimilarity], result of:
            0.011767414 = score(doc=4151,freq=1.0), product of:
              0.058474317 = queryWeight, product of:
                1.0034829 = boost
                3.2198517 = idf(docFreq=4626, maxDocs=42596)
                0.01809753 = queryNorm
              0.20124073 = fieldWeight in 4151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2198517 = idf(docFreq=4626, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.07459721 = weight(abstract_txt:relevance in 4151) [ClassicSimilarity], result of:
            0.07459721 = score(doc=4151,freq=7.0), product of:
              0.09146672 = queryWeight, product of:
                1.0247391 = boost
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.01809753 = queryNorm
              0.8155667 = fieldWeight in 4151, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.010618573 = weight(abstract_txt:with in 4151) [ClassicSimilarity], result of:
            0.010618573 = score(doc=4151,freq=2.0), product of:
              0.047700707 = queryWeight, product of:
                1.0465481 = boost
                2.5185254 = idf(docFreq=9329, maxDocs=42596)
                0.01809753 = queryNorm
              0.2226083 = fieldWeight in 4151, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5185254 = idf(docFreq=9329, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.0328338 = weight(abstract_txt:system in 4151) [ClassicSimilarity], result of:
            0.0328338 = score(doc=4151,freq=6.0), product of:
              0.06377857 = queryWeight, product of:
                1.0480086 = boost
                3.3627198 = idf(docFreq=4010, maxDocs=42596)
                0.01809753 = queryNorm
              0.51480925 = fieldWeight in 4151, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.3627198 = idf(docFreq=4010, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.014251759 = weight(abstract_txt:also in 4151) [ClassicSimilarity], result of:
            0.014251759 = score(doc=4151,freq=1.0), product of:
              0.066439055 = queryWeight, product of:
                1.0696437 = boost
                3.43214 = idf(docFreq=3741, maxDocs=42596)
                0.01809753 = queryNorm
              0.21450876 = fieldWeight in 4151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.43214 = idf(docFreq=3741, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.01477105 = weight(abstract_txt:study in 4151) [ClassicSimilarity], result of:
            0.01477105 = score(doc=4151,freq=1.0), product of:
              0.0680433 = queryWeight, product of:
                1.0824805 = boost
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.01809753 = queryNorm
              0.2170831 = fieldWeight in 4151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.015205185 = weight(abstract_txt:results in 4151) [ClassicSimilarity], result of:
            0.015205185 = score(doc=4151,freq=1.0), product of:
              0.06937009 = queryWeight, product of:
                1.0929834 = boost
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.01809753 = queryNorm
              0.21918935 = fieldWeight in 4151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.055307593 = weight(abstract_txt:overall in 4151) [ClassicSimilarity], result of:
            0.055307593 = score(doc=4151,freq=2.0), product of:
              0.11376113 = queryWeight, product of:
                1.1428224 = boost
                5.50042 = idf(docFreq=472, maxDocs=42596)
                0.01809753 = queryNorm
              0.48617303 = fieldWeight in 4151, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.50042 = idf(docFreq=472, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.035407405 = weight(abstract_txt:analysis in 4151) [ClassicSimilarity], result of:
            0.035407405 = score(doc=4151,freq=4.0), product of:
              0.076775126 = queryWeight, product of:
                1.1498407 = boost
                3.6894662 = idf(docFreq=2892, maxDocs=42596)
                0.01809753 = queryNorm
              0.46118328 = fieldWeight in 4151, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6894662 = idf(docFreq=2892, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.113185756 = weight(abstract_txt:measures in 4151) [ClassicSimilarity], result of:
            0.113185756 = score(doc=4151,freq=4.0), product of:
              0.16660409 = queryWeight, product of:
                1.6938316 = boost
                5.434957 = idf(docFreq=504, maxDocs=42596)
                0.01809753 = queryNorm
              0.6793696 = fieldWeight in 4151, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.434957 = idf(docFreq=504, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.08138408 = weight(abstract_txt:performance in 4151) [ClassicSimilarity], result of:
            0.08138408 = score(doc=4151,freq=3.0), product of:
              0.16198567 = queryWeight, product of:
                1.9285686 = boost
                4.6411138 = idf(docFreq=1116, maxDocs=42596)
                0.01809753 = queryNorm
              0.5024153 = fieldWeight in 4151, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.6411138 = idf(docFreq=1116, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.052045006 = weight(abstract_txt:significant in 4151) [ClassicSimilarity], result of:
            0.052045006 = score(doc=4151,freq=1.0), product of:
              0.17341104 = queryWeight, product of:
                1.9954238 = boost
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.01809753 = queryNorm
              0.3001251 = fieldWeight in 4151, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.114640765 = weight(abstract_txt:user in 4151) [ClassicSimilarity], result of:
            0.114640765 = score(doc=4151,freq=6.0), product of:
              0.20355245 = queryWeight, product of:
                3.0573838 = boost
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.01809753 = queryNorm
              0.5632001 = fieldWeight in 4151, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
          0.45313776 = weight(abstract_txt:satisfaction in 4151) [ClassicSimilarity], result of:
            0.45313776 = score(doc=4151,freq=6.0), product of:
              0.46233582 = queryWeight, product of:
                3.990444 = boost
                6.4020205 = idf(docFreq=191, maxDocs=42596)
                0.01809753 = queryNorm
              0.9801052 = fieldWeight in 4151, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                6.4020205 = idf(docFreq=191, maxDocs=42596)
                0.0625 = fieldNorm(doc=4151)
        0.6 = coord(15/25)
    
  3. Su, L.T.: ¬A comprehensive and systematic model of user evaluation of Web search engines : I. Theory and background (2003) 0.63
    0.62974 = sum of:
      0.62974 = product of:
        1.1245357 = sum of:
          0.028195096 = weight(abstract_txt:relevance in 165) [ClassicSimilarity], result of:
            0.028195096 = score(doc=165,freq=1.0), product of:
              0.09146672 = queryWeight, product of:
                1.0247391 = boost
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.01809753 = queryNorm
              0.30825526 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.018956602 = weight(abstract_txt:system in 165) [ClassicSimilarity], result of:
            0.018956602 = score(doc=165,freq=2.0), product of:
              0.06377857 = queryWeight, product of:
                1.0480086 = boost
                3.3627198 = idf(docFreq=4010, maxDocs=42596)
                0.01809753 = queryNorm
              0.29722524 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.3627198 = idf(docFreq=4010, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.015205185 = weight(abstract_txt:results in 165) [ClassicSimilarity], result of:
            0.015205185 = score(doc=165,freq=1.0), product of:
              0.06937009 = queryWeight, product of:
                1.0929834 = boost
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.01809753 = queryNorm
              0.21918935 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.03942726 = weight(abstract_txt:engine in 165) [ClassicSimilarity], result of:
            0.03942726 = score(doc=165,freq=1.0), product of:
              0.11437869 = queryWeight, product of:
                1.1459202 = boost
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.01809753 = queryNorm
              0.3447081 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.017703703 = weight(abstract_txt:analysis in 165) [ClassicSimilarity], result of:
            0.017703703 = score(doc=165,freq=1.0), product of:
              0.076775126 = queryWeight, product of:
                1.1498407 = boost
                3.6894662 = idf(docFreq=2892, maxDocs=42596)
                0.01809753 = queryNorm
              0.23059164 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6894662 = idf(docFreq=2892, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.11285296 = weight(abstract_txt:undergraduates in 165) [ClassicSimilarity], result of:
            0.11285296 = score(doc=165,freq=1.0), product of:
              0.23058055 = queryWeight, product of:
                1.627021 = boost
                7.8308744 = idf(docFreq=45, maxDocs=42596)
                0.01809753 = queryNorm
              0.48942965 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.8308744 = idf(docFreq=45, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.056592878 = weight(abstract_txt:measures in 165) [ClassicSimilarity], result of:
            0.056592878 = score(doc=165,freq=1.0), product of:
              0.16660409 = queryWeight, product of:
                1.6938316 = boost
                5.434957 = idf(docFreq=504, maxDocs=42596)
                0.01809753 = queryNorm
              0.3396848 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.434957 = idf(docFreq=504, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.06138315 = weight(abstract_txt:criteria in 165) [ClassicSimilarity], result of:
            0.06138315 = score(doc=165,freq=1.0), product of:
              0.17587762 = queryWeight, product of:
                1.7403344 = boost
                5.5841694 = idf(docFreq=434, maxDocs=42596)
                0.01809753 = queryNorm
              0.3490106 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5841694 = idf(docFreq=434, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.08403814 = weight(abstract_txt:four in 165) [ClassicSimilarity], result of:
            0.08403814 = score(doc=165,freq=1.0), product of:
              0.25710568 = queryWeight, product of:
                2.716487 = boost
                5.229796 = idf(docFreq=619, maxDocs=42596)
                0.01809753 = queryNorm
              0.32686225 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.229796 = idf(docFreq=619, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.110661894 = weight(abstract_txt:evaluation in 165) [ClassicSimilarity], result of:
            0.110661894 = score(doc=165,freq=3.0), product of:
              0.2275863 = queryWeight, product of:
                2.7997255 = boost
                4.4917064 = idf(docFreq=1296, maxDocs=42596)
                0.01809753 = queryNorm
              0.48624146 = fieldWeight in 165, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.4917064 = idf(docFreq=1296, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.06618787 = weight(abstract_txt:user in 165) [ClassicSimilarity], result of:
            0.06618787 = score(doc=165,freq=2.0), product of:
              0.20355245 = queryWeight, product of:
                3.0573838 = boost
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.01809753 = queryNorm
              0.32516372 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.08094919 = weight(abstract_txt:search in 165) [ClassicSimilarity], result of:
            0.08094919 = score(doc=165,freq=2.0), product of:
              0.25076622 = queryWeight, product of:
                3.7940347 = boost
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.01809753 = queryNorm
              0.32280737 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.18499272 = weight(abstract_txt:satisfaction in 165) [ClassicSimilarity], result of:
            0.18499272 = score(doc=165,freq=1.0), product of:
              0.46233582 = queryWeight, product of:
                3.990444 = boost
                6.4020205 = idf(docFreq=191, maxDocs=42596)
                0.01809753 = queryNorm
              0.40012628 = fieldWeight in 165, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.4020205 = idf(docFreq=191, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
          0.24738902 = weight(abstract_txt:engines in 165) [ClassicSimilarity], result of:
            0.24738902 = score(doc=165,freq=2.0), product of:
              0.52809775 = queryWeight, product of:
                5.5058446 = boost
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.01809753 = queryNorm
              0.46845308 = fieldWeight in 165, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.0625 = fieldNorm(doc=165)
        0.56 = coord(14/25)
    
  4. Spink, A.; Jansen, B.J.; Blakely, C.; Koshman, S.: ¬A study of results overlap and uniqueness among major Web search engines (2006) 0.53
    0.5311777 = sum of:
      0.5311777 = product of:
        1.207222 = sum of:
          0.010296487 = weight(abstract_txt:these in 2173) [ClassicSimilarity], result of:
            0.010296487 = score(doc=2173,freq=1.0), product of:
              0.058474317 = queryWeight, product of:
                1.0034829 = boost
                3.2198517 = idf(docFreq=4626, maxDocs=42596)
                0.01809753 = queryNorm
              0.17608564 = fieldWeight in 2173, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2198517 = idf(docFreq=4626, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.0065699066 = weight(abstract_txt:with in 2173) [ClassicSimilarity], result of:
            0.0065699066 = score(doc=2173,freq=1.0), product of:
              0.047700707 = queryWeight, product of:
                1.0465481 = boost
                2.5185254 = idf(docFreq=9329, maxDocs=42596)
                0.01809753 = queryNorm
              0.13773185 = fieldWeight in 2173, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.5185254 = idf(docFreq=9329, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.028900439 = weight(abstract_txt:study in 2173) [ClassicSimilarity], result of:
            0.028900439 = score(doc=2173,freq=5.0), product of:
              0.0680433 = queryWeight, product of:
                1.0824805 = boost
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.01809753 = queryNorm
              0.424736 = fieldWeight in 2173, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.03991361 = weight(abstract_txt:results in 2173) [ClassicSimilarity], result of:
            0.03991361 = score(doc=2173,freq=9.0), product of:
              0.06937009 = queryWeight, product of:
                1.0929834 = boost
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.01809753 = queryNorm
              0.57537204 = fieldWeight in 2173, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.04878875 = weight(abstract_txt:engine in 2173) [ClassicSimilarity], result of:
            0.04878875 = score(doc=2173,freq=2.0), product of:
              0.11437869 = queryWeight, product of:
                1.1459202 = boost
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.01809753 = queryNorm
              0.42655453 = fieldWeight in 2173, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.04111373 = weight(abstract_txt:performance in 2173) [ClassicSimilarity], result of:
            0.04111373 = score(doc=2173,freq=1.0), product of:
              0.16198567 = queryWeight, product of:
                1.9285686 = boost
                4.6411138 = idf(docFreq=1116, maxDocs=42596)
                0.01809753 = queryNorm
              0.2538109 = fieldWeight in 2173, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6411138 = idf(docFreq=1116, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.06440241 = weight(abstract_txt:significant in 2173) [ClassicSimilarity], result of:
            0.06440241 = score(doc=2173,freq=2.0), product of:
              0.17341104 = queryWeight, product of:
                1.9954238 = boost
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.01809753 = queryNorm
              0.37138587 = fieldWeight in 2173, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.14706674 = weight(abstract_txt:four in 2173) [ClassicSimilarity], result of:
            0.14706674 = score(doc=2173,freq=4.0), product of:
              0.25710568 = queryWeight, product of:
                2.716487 = boost
                5.229796 = idf(docFreq=619, maxDocs=42596)
                0.01809753 = queryNorm
              0.5720089 = fieldWeight in 2173, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.229796 = idf(docFreq=619, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.040951658 = weight(abstract_txt:user in 2173) [ClassicSimilarity], result of:
            0.040951658 = score(doc=2173,freq=1.0), product of:
              0.20355245 = queryWeight, product of:
                3.0573838 = boost
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.01809753 = queryNorm
              0.2011848 = fieldWeight in 2173, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.2065047 = weight(abstract_txt:search in 2173) [ClassicSimilarity], result of:
            0.2065047 = score(doc=2173,freq=17.0), product of:
              0.25076622 = queryWeight, product of:
                3.7940347 = boost
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.01809753 = queryNorm
              0.8234949 = fieldWeight in 2173, product of:
                4.1231055 = tf(freq=17.0), with freq of:
                  17.0 = termFreq=17.0
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
          0.5727136 = weight(abstract_txt:engines in 2173) [ClassicSimilarity], result of:
            0.5727136 = score(doc=2173,freq=14.0), product of:
              0.52809775 = queryWeight, product of:
                5.5058446 = boost
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.01809753 = queryNorm
              1.0844841 = fieldWeight in 2173, product of:
                3.7416575 = tf(freq=14.0), with freq of:
                  14.0 = termFreq=14.0
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.0546875 = fieldNorm(doc=2173)
        0.44 = coord(11/25)
    
  5. Can, F.; Nuray, R.; Sevdik, A.B.: Automatic performance evaluation of Web search engines (2004) 0.47
    0.47131225 = sum of:
      0.47131225 = product of:
        1.0711643 = sum of:
          0.02058619 = weight(abstract_txt:based in 3571) [ClassicSimilarity], result of:
            0.02058619 = score(doc=3571,freq=2.0), product of:
              0.058069106 = queryWeight, product of:
                3.2086759 = idf(docFreq=4678, maxDocs=42596)
                0.01809753 = queryNorm
              0.35451192 = fieldWeight in 3571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2086759 = idf(docFreq=4678, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.049842358 = weight(abstract_txt:relevance in 3571) [ClassicSimilarity], result of:
            0.049842358 = score(doc=3571,freq=2.0), product of:
              0.09146672 = queryWeight, product of:
                1.0247391 = boost
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.01809753 = queryNorm
              0.5449234 = fieldWeight in 3571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.932084 = idf(docFreq=834, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.013273217 = weight(abstract_txt:with in 3571) [ClassicSimilarity], result of:
            0.013273217 = score(doc=3571,freq=2.0), product of:
              0.047700707 = queryWeight, product of:
                1.0465481 = boost
                2.5185254 = idf(docFreq=9329, maxDocs=42596)
                0.01809753 = queryNorm
              0.27826038 = fieldWeight in 3571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5185254 = idf(docFreq=9329, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.018463813 = weight(abstract_txt:study in 3571) [ClassicSimilarity], result of:
            0.018463813 = score(doc=3571,freq=1.0), product of:
              0.0680433 = queryWeight, product of:
                1.0824805 = boost
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.01809753 = queryNorm
              0.27135387 = fieldWeight in 3571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4733295 = idf(docFreq=3590, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.019006481 = weight(abstract_txt:results in 3571) [ClassicSimilarity], result of:
            0.019006481 = score(doc=3571,freq=1.0), product of:
              0.06937009 = queryWeight, product of:
                1.0929834 = boost
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.01809753 = queryNorm
              0.2739867 = fieldWeight in 3571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5070295 = idf(docFreq=3471, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.04928408 = weight(abstract_txt:engine in 3571) [ClassicSimilarity], result of:
            0.04928408 = score(doc=3571,freq=1.0), product of:
              0.11437869 = queryWeight, product of:
                1.1459202 = boost
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.01809753 = queryNorm
              0.43088514 = fieldWeight in 3571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.51533 = idf(docFreq=465, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.06505626 = weight(abstract_txt:significant in 3571) [ClassicSimilarity], result of:
            0.06505626 = score(doc=3571,freq=1.0), product of:
              0.17341104 = queryWeight, product of:
                1.9954238 = boost
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.01809753 = queryNorm
              0.37515637 = fieldWeight in 3571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8020015 = idf(docFreq=950, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.11294383 = weight(abstract_txt:evaluation in 3571) [ClassicSimilarity], result of:
            0.11294383 = score(doc=3571,freq=2.0), product of:
              0.2275863 = queryWeight, product of:
                2.7997255 = boost
                4.4917064 = idf(docFreq=1296, maxDocs=42596)
                0.01809753 = queryNorm
              0.49626812 = fieldWeight in 3571, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4917064 = idf(docFreq=1296, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.058502372 = weight(abstract_txt:user in 3571) [ClassicSimilarity], result of:
            0.058502372 = score(doc=3571,freq=1.0), product of:
              0.20355245 = queryWeight, product of:
                3.0573838 = boost
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.01809753 = queryNorm
              0.28740686 = fieldWeight in 3571, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6788077 = idf(docFreq=2923, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.17526013 = weight(abstract_txt:search in 3571) [ClassicSimilarity], result of:
            0.17526013 = score(doc=3571,freq=6.0), product of:
              0.25076622 = queryWeight, product of:
                3.7940347 = boost
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.01809753 = queryNorm
              0.6988985 = fieldWeight in 3571, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.6521485 = idf(docFreq=3002, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
          0.48894545 = weight(abstract_txt:engines in 3571) [ClassicSimilarity], result of:
            0.48894545 = score(doc=3571,freq=5.0), product of:
              0.52809775 = queryWeight, product of:
                5.5058446 = boost
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.01809753 = queryNorm
              0.92586166 = fieldWeight in 3571, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.2999415 = idf(docFreq=577, maxDocs=42596)
                0.078125 = fieldNorm(doc=3571)
        0.44 = coord(11/25)