Document (#27537)

Author
Vaughan, L.
Title
New measurements for search engine evaluation proposed and tested
Source
Information processing and management. 40(2004) no.4, S.677-691
Year
2004
Abstract
A set of measurements is proposed for evaluating Web search engine performance. Some measurements are adapted from the concepts of recall and precision, which are commonly used in evaluating traditional information retrieval systems. Others are newly developed to evaluate search engine stability, an issue unique to Web information retrieval systems. An experiment was conducted to test these new measurements by applying them to a performance comparison of three commercial search engines: Google, AltaVista, and Teoma. Twenty-four subjects ranked four sets of Web pages and their rankings were used as benchmarks against which to compare search engine performance. Results show that the proposed measurements are able to distinguish search engine performance very well.
Theme
Suchmaschinen

Similar documents (author)

  1. Vaughan, L.Q.: Information search patterns of business communities : a comparison between small and medium-sized businesses (1997) 5.09
    5.085331 = sum of:
      5.085331 = weight(author_txt:vaughan in 2422) [ClassicSimilarity], result of:
        5.085331 = fieldWeight in 2422, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.13653 = idf(docFreq=33, maxDocs=42740)
          0.625 = fieldNorm(doc=2422)
    
  2. Vaughan, J.: Considerations in the choice of an Interet search tool (1999) 5.09
    5.085331 = sum of:
      5.085331 = weight(author_txt:vaughan in 283) [ClassicSimilarity], result of:
        5.085331 = fieldWeight in 283, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.13653 = idf(docFreq=33, maxDocs=42740)
          0.625 = fieldNorm(doc=283)
    
  3. Vaughan, K.T.L.: impacts of electronic equivalents on print chemistry journal use : Changing use patterns of print journals in the digital age (2003) 5.09
    5.085331 = sum of:
      5.085331 = weight(author_txt:vaughan in 2874) [ClassicSimilarity], result of:
        5.085331 = fieldWeight in 2874, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.13653 = idf(docFreq=33, maxDocs=42740)
          0.625 = fieldNorm(doc=2874)
    
  4. Vaughan, L.: Statistical methods for the information professional : a practical, painless approach to understanding, using, and interpreting statistics (2001) 5.09
    5.085331 = sum of:
      5.085331 = weight(author_txt:vaughan in 685) [ClassicSimilarity], result of:
        5.085331 = fieldWeight in 685, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.13653 = idf(docFreq=33, maxDocs=42740)
          0.625 = fieldNorm(doc=685)
    
  5. Vaughan, L.: Visualizing linguistic and cultural differences using Web co-link data (2006) 5.09
    5.085331 = sum of:
      5.085331 = weight(author_txt:vaughan in 1310) [ClassicSimilarity], result of:
        5.085331 = fieldWeight in 1310, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.13653 = idf(docFreq=33, maxDocs=42740)
          0.625 = fieldNorm(doc=1310)
    

Similar documents (content)

  1. Truran, M.; Schmakeit, J.-F.; Ashman, H.: ¬The effect of user intent on the stability of search engine results (2011) 0.34
    0.3392356 = sum of:
      0.3392356 = product of:
        1.2115557 = sum of:
          0.0422721 = weight(abstract_txt:commercial in 1479) [ClassicSimilarity], result of:
            0.0422721 = score(doc=1479,freq=1.0), product of:
              0.07827817 = queryWeight, product of:
                5.7602572 = idf(docFreq=365, maxDocs=42740)
                0.013589353 = queryNorm
              0.5400241 = fieldWeight in 1479, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.7602572 = idf(docFreq=365, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
          0.011166409 = weight(abstract_txt:which in 1479) [ClassicSimilarity], result of:
            0.011166409 = score(doc=1479,freq=1.0), product of:
              0.040602908 = queryWeight, product of:
                1.0185286 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.013589353 = queryNorm
              0.275015 = fieldWeight in 1479, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
          0.09853454 = weight(abstract_txt:stability in 1479) [ClassicSimilarity], result of:
            0.09853454 = score(doc=1479,freq=1.0), product of:
              0.13761437 = queryWeight, product of:
                1.3259023 = boost
                7.637539 = idf(docFreq=55, maxDocs=42740)
                0.013589353 = queryNorm
              0.7160193 = fieldWeight in 1479, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.637539 = idf(docFreq=55, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
          0.063219436 = weight(abstract_txt:four in 1479) [ClassicSimilarity], result of:
            0.063219436 = score(doc=1479,freq=1.0), product of:
              0.12897786 = queryWeight, product of:
                1.8153161 = boost
                5.228344 = idf(docFreq=622, maxDocs=42740)
                0.013589353 = queryNorm
              0.49015725 = fieldWeight in 1479, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.228344 = idf(docFreq=622, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
          0.12922189 = weight(abstract_txt:search in 1479) [ClassicSimilarity], result of:
            0.12922189 = score(doc=1479,freq=4.0), product of:
              0.18873796 = queryWeight, product of:
                3.803513 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.013589353 = queryNorm
              0.68466294 = fieldWeight in 1479, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
          0.3215616 = weight(abstract_txt:engine in 1479) [ClassicSimilarity], result of:
            0.3215616 = score(doc=1479,freq=3.0), product of:
              0.3589747 = queryWeight, product of:
                4.788468 = boost
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.013589353 = queryNorm
              0.89577794 = fieldWeight in 1479, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
          0.54557973 = weight(abstract_txt:measurements in 1479) [ClassicSimilarity], result of:
            0.54557973 = score(doc=1479,freq=1.0), product of:
              0.73649013 = queryWeight, product of:
                6.8588 = boost
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.013589353 = queryNorm
              0.74078345 = fieldWeight in 1479, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.09375 = fieldNorm(doc=1479)
        0.28 = coord(7/25)
    
  2. Mettrop, W.; Nieuwenhuysen, P.: Internet search engines : fluctuations in document accessibility (2001) 0.23
    0.22715363 = sum of:
      0.22715363 = product of:
        0.81126297 = sum of:
          0.0074442727 = weight(abstract_txt:which in 482) [ClassicSimilarity], result of:
            0.0074442727 = score(doc=482,freq=1.0), product of:
              0.040602908 = queryWeight, product of:
                1.0185286 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.013589353 = queryNorm
              0.18334334 = fieldWeight in 482, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
          0.012265972 = weight(abstract_txt:retrieval in 482) [ClassicSimilarity], result of:
            0.012265972 = score(doc=482,freq=1.0), product of:
              0.056642655 = queryWeight, product of:
                1.2030023 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.013589353 = queryNorm
              0.21655008 = fieldWeight in 482, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
          0.058120888 = weight(abstract_txt:altavista in 482) [ClassicSimilarity], result of:
            0.058120888 = score(doc=482,freq=1.0), product of:
              0.12682955 = queryWeight, product of:
                1.2728871 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.013589353 = queryNorm
              0.45825982 = fieldWeight in 482, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
          0.059021294 = weight(abstract_txt:performance in 482) [ClassicSimilarity], result of:
            0.059021294 = score(doc=482,freq=1.0), product of:
              0.20340335 = queryWeight, product of:
                3.223953 = boost
                4.6426997 = idf(docFreq=1118, maxDocs=42740)
                0.013589353 = queryNorm
              0.29016873 = fieldWeight in 482, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6426997 = idf(docFreq=1118, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
          0.09631631 = weight(abstract_txt:search in 482) [ClassicSimilarity], result of:
            0.09631631 = score(doc=482,freq=5.0), product of:
              0.18873796 = queryWeight, product of:
                3.803513 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.013589353 = queryNorm
              0.5103176 = fieldWeight in 482, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
          0.21437442 = weight(abstract_txt:engine in 482) [ClassicSimilarity], result of:
            0.21437442 = score(doc=482,freq=3.0), product of:
              0.3589747 = queryWeight, product of:
                4.788468 = boost
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.013589353 = queryNorm
              0.5971853 = fieldWeight in 482, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
          0.36371982 = weight(abstract_txt:measurements in 482) [ClassicSimilarity], result of:
            0.36371982 = score(doc=482,freq=1.0), product of:
              0.73649013 = queryWeight, product of:
                6.8588 = boost
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.013589353 = queryNorm
              0.49385566 = fieldWeight in 482, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.0625 = fieldNorm(doc=482)
        0.28 = coord(7/25)
    
  3. Petric, K.; Petric, T.; Krisper, M.; Rajkovic, V.: User profiling on a pilot digital library with the final result of a new adaptive knowledge management solution (2011) 0.22
    0.22444734 = sum of:
      0.22444734 = product of:
        0.9351973 = sum of:
          0.009305341 = weight(abstract_txt:which in 1561) [ClassicSimilarity], result of:
            0.009305341 = score(doc=1561,freq=1.0), product of:
              0.040602908 = queryWeight, product of:
                1.0185286 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.013589353 = queryNorm
              0.22917917 = fieldWeight in 1561, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.078125 = fieldNorm(doc=1561)
          0.02168338 = weight(abstract_txt:retrieval in 1561) [ClassicSimilarity], result of:
            0.02168338 = score(doc=1561,freq=2.0), product of:
              0.056642655 = queryWeight, product of:
                1.2030023 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.013589353 = queryNorm
              0.3828101 = fieldWeight in 1561, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.078125 = fieldNorm(doc=1561)
          0.05268286 = weight(abstract_txt:four in 1561) [ClassicSimilarity], result of:
            0.05268286 = score(doc=1561,freq=1.0), product of:
              0.12897786 = queryWeight, product of:
                1.8153161 = boost
                5.228344 = idf(docFreq=622, maxDocs=42740)
                0.013589353 = queryNorm
              0.40846437 = fieldWeight in 1561, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.228344 = idf(docFreq=622, maxDocs=42740)
                0.078125 = fieldNorm(doc=1561)
          0.053842448 = weight(abstract_txt:search in 1561) [ClassicSimilarity], result of:
            0.053842448 = score(doc=1561,freq=1.0), product of:
              0.18873796 = queryWeight, product of:
                3.803513 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.013589353 = queryNorm
              0.2852762 = fieldWeight in 1561, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.078125 = fieldNorm(doc=1561)
          0.15471141 = weight(abstract_txt:engine in 1561) [ClassicSimilarity], result of:
            0.15471141 = score(doc=1561,freq=1.0), product of:
              0.3589747 = queryWeight, product of:
                4.788468 = boost
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.013589353 = queryNorm
              0.43098137 = fieldWeight in 1561, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.078125 = fieldNorm(doc=1561)
          0.6429719 = weight(abstract_txt:measurements in 1561) [ClassicSimilarity], result of:
            0.6429719 = score(doc=1561,freq=2.0), product of:
              0.73649013 = queryWeight, product of:
                6.8588 = boost
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.013589353 = queryNorm
              0.8730217 = fieldWeight in 1561, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.078125 = fieldNorm(doc=1561)
        0.24 = coord(6/25)
    
  4. Thelwall, M.: Assessing web search engines : a webometric approach (2011) 0.20
    0.19783147 = sum of:
      0.19783147 = product of:
        0.8242978 = sum of:
          0.01315974 = weight(abstract_txt:which in 2011) [ClassicSimilarity], result of:
            0.01315974 = score(doc=2011,freq=2.0), product of:
              0.040602908 = queryWeight, product of:
                1.0185286 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.013589353 = queryNorm
              0.3241083 = fieldWeight in 2011, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.078125 = fieldNorm(doc=2011)
          0.014675915 = weight(abstract_txt:systems in 2011) [ClassicSimilarity], result of:
            0.014675915 = score(doc=2011,freq=1.0), product of:
              0.0550139 = queryWeight, product of:
                1.18558 = boost
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.013589353 = queryNorm
              0.2667674 = fieldWeight in 2011, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.078125 = fieldNorm(doc=2011)
          0.015332465 = weight(abstract_txt:retrieval in 2011) [ClassicSimilarity], result of:
            0.015332465 = score(doc=2011,freq=1.0), product of:
              0.056642655 = queryWeight, product of:
                1.2030023 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.013589353 = queryNorm
              0.2706876 = fieldWeight in 2011, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.078125 = fieldNorm(doc=2011)
          0.107684895 = weight(abstract_txt:search in 2011) [ClassicSimilarity], result of:
            0.107684895 = score(doc=2011,freq=4.0), product of:
              0.18873796 = queryWeight, product of:
                3.803513 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.013589353 = queryNorm
              0.5705524 = fieldWeight in 2011, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.078125 = fieldNorm(doc=2011)
          0.21879497 = weight(abstract_txt:engine in 2011) [ClassicSimilarity], result of:
            0.21879497 = score(doc=2011,freq=2.0), product of:
              0.3589747 = queryWeight, product of:
                4.788468 = boost
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.013589353 = queryNorm
              0.6094997 = fieldWeight in 2011, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.078125 = fieldNorm(doc=2011)
          0.45464978 = weight(abstract_txt:measurements in 2011) [ClassicSimilarity], result of:
            0.45464978 = score(doc=2011,freq=1.0), product of:
              0.73649013 = queryWeight, product of:
                6.8588 = boost
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.013589353 = queryNorm
              0.6173196 = fieldWeight in 2011, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.9016905 = idf(docFreq=42, maxDocs=42740)
                0.078125 = fieldNorm(doc=2011)
        0.24 = coord(6/25)
    
  5. Bar-Ilan, J.: Methods for measuring search engine performance over time (2002) 0.17
    0.1662961 = sum of:
      0.1662961 = product of:
        0.83148044 = sum of:
          0.013027477 = weight(abstract_txt:which in 1306) [ClassicSimilarity], result of:
            0.013027477 = score(doc=1306,freq=1.0), product of:
              0.040602908 = queryWeight, product of:
                1.0185286 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.013589353 = queryNorm
              0.32085085 = fieldWeight in 1306, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.109375 = fieldNorm(doc=1306)
          0.14646862 = weight(abstract_txt:evaluating in 1306) [ClassicSimilarity], result of:
            0.14646862 = score(doc=1306,freq=2.0), product of:
              0.16173474 = queryWeight, product of:
                2.0328078 = boost
                5.8547482 = idf(docFreq=332, maxDocs=42740)
                0.013589353 = queryNorm
              0.90561014 = fieldWeight in 1306, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.8547482 = idf(docFreq=332, maxDocs=42740)
                0.109375 = fieldNorm(doc=1306)
          0.14607024 = weight(abstract_txt:performance in 1306) [ClassicSimilarity], result of:
            0.14607024 = score(doc=1306,freq=2.0), product of:
              0.20340335 = queryWeight, product of:
                3.223953 = boost
                4.6426997 = idf(docFreq=1118, maxDocs=42740)
                0.013589353 = queryNorm
              0.71813095 = fieldWeight in 1306, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6426997 = idf(docFreq=1118, maxDocs=42740)
                0.109375 = fieldNorm(doc=1306)
          0.15075886 = weight(abstract_txt:search in 1306) [ClassicSimilarity], result of:
            0.15075886 = score(doc=1306,freq=4.0), product of:
              0.18873796 = queryWeight, product of:
                3.803513 = boost
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.013589353 = queryNorm
              0.7987734 = fieldWeight in 1306, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6515355 = idf(docFreq=3014, maxDocs=42740)
                0.109375 = fieldNorm(doc=1306)
          0.37515524 = weight(abstract_txt:engine in 1306) [ClassicSimilarity], result of:
            0.37515524 = score(doc=1306,freq=3.0), product of:
              0.3589747 = queryWeight, product of:
                4.788468 = boost
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.013589353 = queryNorm
              1.0450743 = fieldWeight in 1306, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.5165615 = idf(docFreq=466, maxDocs=42740)
                0.109375 = fieldNorm(doc=1306)
        0.2 = coord(5/25)