Document (#16329)

Author
Shaw, W.M.
Burgin, R.
Howell, P.
Title
Performance standards and evaluations in IR test collections : vector-space and other retrieval models
Source
Information processing and management. 33(1997) no.1, S.15-36
Year
1997
Abstract
Computes low performance standards for each query and for the group of queries in 13 traditional and 4 TREC test collections. Predicted by the hypergeometric distribution, the standards represent the highest level of retrieval effectiveness attributable to chance. Compares operational levels of performance for vector-space, ad-hoc-feature-based, probabilistic, and other retrieval models to the standards. The effectiveness of these techniques in small, traditional test collections, can be explained by retrieving a few more relevant documents for most queries than expected by chance. The effectiveness of retrieval techniques in the larger TREC test collections can only be explained by retrieving many more relevant documents for most queries than expected by chance. The discrepancy between deviations form chance in traditional and TREC test collections is due to a decrease in performance standards for large test collections, not to an increase in operational performance. The next generation of information retrieval systems would be enhanced by abandoning uninformative performance summaries and focusing on effectiveness and improvements in effectiveness of individual queries
Theme
Retrievalstudien
Object
TREC

Similar documents (author)

  1. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 4.68 
    4.6815257 = sum of:
  4.6815257 = sum of:
    1.7237445 = weight(author_txt:shaw in 7256) [ClassicSimilarity], result of:
      1.7237445 = score(doc=7256,freq=1.0), product of:
        0.5721979 = queryWeight, product of:
          8.033325 = idf(docFreq=38, maxDocs=44218)
          0.07122803 = queryNorm
        3.012497 = fieldWeight in 7256, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.033325 = idf(docFreq=38, maxDocs=44218)
          0.375 = fieldNorm(doc=7256)
    2.9577813 = weight(author_txt:burgin in 7256) [ClassicSimilarity], result of:
      2.9577813 = score(doc=7256,freq=1.0), product of:
        0.82011557 = queryWeight, product of:
          1.1971936 = boost
          9.617446 = idf(docFreq=7, maxDocs=44218)
          0.07122803 = queryNorm
        3.606542 = fieldWeight in 7256, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.617446 = idf(docFreq=7, maxDocs=44218)
          0.375 = fieldNorm(doc=7256)
  2. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 2.46 
    2.464818 = sum of:
  2.464818 = product of:
    4.929636 = sum of:
      4.929636 = weight(author_txt:burgin in 3308) [ClassicSimilarity], result of:
        4.929636 = score(doc=3308,freq=1.0), product of:
          0.82011557 = queryWeight, product of:
            1.1971936 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07122803 = queryNorm
          6.010904 = fieldWeight in 3308, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=3308)
    0.5 = coord(1/2)
  3. Burgin, R.: ¬The retrieval effectiveness of 5 clustering algorithms as a function of indexing exhaustivity (1995) 2.46 
    2.464818 = sum of:
  2.464818 = product of:
    4.929636 = sum of:
      4.929636 = weight(author_txt:burgin in 3365) [ClassicSimilarity], result of:
        4.929636 = score(doc=3365,freq=1.0), product of:
          0.82011557 = queryWeight, product of:
            1.1971936 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07122803 = queryNorm
          6.010904 = fieldWeight in 3365, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=3365)
    0.5 = coord(1/2)
  4. Burgin, M.: Terminological and teleological systems (1996) 2.46 
    2.464818 = sum of:
  2.464818 = product of:
    4.929636 = sum of:
      4.929636 = weight(author_txt:burgin in 6323) [ClassicSimilarity], result of:
        4.929636 = score(doc=6323,freq=1.0), product of:
          0.82011557 = queryWeight, product of:
            1.1971936 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07122803 = queryNorm
          6.010904 = fieldWeight in 6323, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=6323)
    0.5 = coord(1/2)
  5. Burgin, R.: ¬The Monte Carlo method and the evaluation of retrieval system performance (1999) 2.46 
    2.464818 = sum of:
  2.464818 = product of:
    4.929636 = sum of:
      4.929636 = weight(author_txt:burgin in 2946) [ClassicSimilarity], result of:
        4.929636 = score(doc=2946,freq=1.0), product of:
          0.82011557 = queryWeight, product of:
            1.1971936 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07122803 = queryNorm
          6.010904 = fieldWeight in 2946, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=2946)
    0.5 = coord(1/2)

Similar documents (content)

  1. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 0.55 
    0.54531187 = sum of:
  0.54531187 = product of:
    1.3632796 = sum of:
      0.13742724 = weight(abstract_txt:computes in 7256) [ClassicSimilarity], result of:
        0.13742724 = score(doc=7256,freq=1.0), product of:
          0.16307901 = queryWeight, product of:
            1.0905335 = boost
            8.988837 = idf(docFreq=14, maxDocs=44218)
            0.016636252 = queryNorm
          0.84270346 = fieldWeight in 7256, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.988837 = idf(docFreq=14, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.035175767 = weight(abstract_txt:techniques in 7256) [ClassicSimilarity], result of:
        0.035175767 = score(doc=7256,freq=1.0), product of:
          0.08283017 = queryWeight, product of:
            1.0991305 = boost
            4.5298495 = idf(docFreq=1295, maxDocs=44218)
            0.016636252 = queryNorm
          0.42467338 = fieldWeight in 7256, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.5298495 = idf(docFreq=1295, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.10603315 = weight(abstract_txt:explained in 7256) [ClassicSimilarity], result of:
        0.10603315 = score(doc=7256,freq=1.0), product of:
          0.17284384 = queryWeight, product of:
            1.5877494 = boost
            6.543596 = idf(docFreq=172, maxDocs=44218)
            0.016636252 = queryNorm
          0.6134621 = fieldWeight in 7256, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.543596 = idf(docFreq=172, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.19139768 = weight(abstract_txt:operational in 7256) [ClassicSimilarity], result of:
        0.19139768 = score(doc=7256,freq=3.0), product of:
          0.1776684 = queryWeight, product of:
            1.6097561 = boost
            6.634292 = idf(docFreq=157, maxDocs=44218)
            0.016636252 = queryNorm
          1.0772748 = fieldWeight in 7256, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.634292 = idf(docFreq=157, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.07941069 = weight(abstract_txt:retrieval in 7256) [ClassicSimilarity], result of:
        0.07941069 = score(doc=7256,freq=4.0), product of:
          0.12187243 = queryWeight, product of:
            2.1080346 = boost
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.016636252 = queryNorm
          0.6515886 = fieldWeight in 7256, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.10078925 = weight(abstract_txt:queries in 7256) [ClassicSimilarity], result of:
        0.10078925 = score(doc=7256,freq=1.0), product of:
          0.21052918 = queryWeight, product of:
            2.4781418 = boost
            5.106586 = idf(docFreq=727, maxDocs=44218)
            0.016636252 = queryNorm
          0.47874242 = fieldWeight in 7256, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.106586 = idf(docFreq=727, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.18124622 = weight(abstract_txt:standards in 7256) [ClassicSimilarity], result of:
        0.18124622 = score(doc=7256,freq=3.0), product of:
          0.23252968 = queryWeight, product of:
            2.9118178 = boost
            4.800193 = idf(docFreq=988, maxDocs=44218)
            0.016636252 = queryNorm
          0.7794541 = fieldWeight in 7256, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.800193 = idf(docFreq=988, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.15940052 = weight(abstract_txt:performance in 7256) [ClassicSimilarity], result of:
        0.15940052 = score(doc=7256,freq=2.0), product of:
          0.25964686 = queryWeight, product of:
            3.3705995 = boost
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.016636252 = queryNorm
          0.61391276 = fieldWeight in 7256, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.16603832 = weight(abstract_txt:collections in 7256) [ClassicSimilarity], result of:
        0.16603832 = score(doc=7256,freq=2.0), product of:
          0.26680592 = queryWeight, product of:
            3.4167511 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.016636252 = queryNorm
          0.62231874 = fieldWeight in 7256, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
      0.20636082 = weight(abstract_txt:test in 7256) [ClassicSimilarity], result of:
        0.20636082 = score(doc=7256,freq=2.0), product of:
          0.3084192 = queryWeight, product of:
            3.673553 = boost
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.016636252 = queryNorm
          0.669092 = fieldWeight in 7256, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.09375 = fieldNorm(doc=7256)
    0.4 = coord(10/25)
  2. TREC: experiment and evaluation in information retrieval (2005) 0.36 
    0.35953403 = sum of:
  0.35953403 = product of:
    0.99870557 = sum of:
      0.017660562 = weight(abstract_txt:documents in 636) [ClassicSimilarity], result of:
        0.017660562 = score(doc=636,freq=1.0), product of:
          0.06856303 = queryWeight, product of:
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.016636252 = queryNorm
          0.2575814 = fieldWeight in 636, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.023450512 = weight(abstract_txt:techniques in 636) [ClassicSimilarity], result of:
        0.023450512 = score(doc=636,freq=1.0), product of:
          0.08283017 = queryWeight, product of:
            1.0991305 = boost
            4.5298495 = idf(docFreq=1295, maxDocs=44218)
            0.016636252 = queryNorm
          0.2831156 = fieldWeight in 636, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.5298495 = idf(docFreq=1295, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.065631844 = weight(abstract_txt:retrieving in 636) [ClassicSimilarity], result of:
        0.065631844 = score(doc=636,freq=1.0), product of:
          0.16449904 = queryWeight, product of:
            1.5489475 = boost
            6.3836813 = idf(docFreq=202, maxDocs=44218)
            0.016636252 = queryNorm
          0.39898008 = fieldWeight in 636, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.3836813 = idf(docFreq=202, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.09169557 = weight(abstract_txt:retrieval in 636) [ClassicSimilarity], result of:
        0.09169557 = score(doc=636,freq=12.0), product of:
          0.12187243 = queryWeight, product of:
            2.1080346 = boost
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.016636252 = queryNorm
          0.7523897 = fieldWeight in 636, product of:
            3.4641016 = tf(freq=12.0), with freq of:
              12.0 = termFreq=12.0
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.32188335 = weight(abstract_txt:trec in 636) [ClassicSimilarity], result of:
        0.32188335 = score(doc=636,freq=8.0), product of:
          0.2717814 = queryWeight, product of:
            2.4384313 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.016636252 = queryNorm
          1.1843464 = fieldWeight in 636, product of:
            2.828427 = tf(freq=8.0), with freq of:
              8.0 = termFreq=8.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.06976171 = weight(abstract_txt:standards in 636) [ClassicSimilarity], result of:
        0.06976171 = score(doc=636,freq=1.0), product of:
          0.23252968 = queryWeight, product of:
            2.9118178 = boost
            4.800193 = idf(docFreq=988, maxDocs=44218)
            0.016636252 = queryNorm
          0.30001205 = fieldWeight in 636, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.800193 = idf(docFreq=988, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.083586685 = weight(abstract_txt:effectiveness in 636) [ClassicSimilarity], result of:
        0.083586685 = score(doc=636,freq=1.0), product of:
          0.26231617 = queryWeight, product of:
            3.092698 = boost
            5.098378 = idf(docFreq=733, maxDocs=44218)
            0.016636252 = queryNorm
          0.31864864 = fieldWeight in 636, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.098378 = idf(docFreq=733, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.15654244 = weight(abstract_txt:collections in 636) [ClassicSimilarity], result of:
        0.15654244 = score(doc=636,freq=4.0), product of:
          0.26680592 = queryWeight, product of:
            3.4167511 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.016636252 = queryNorm
          0.58672774 = fieldWeight in 636, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
      0.1684929 = weight(abstract_txt:test in 636) [ClassicSimilarity], result of:
        0.1684929 = score(doc=636,freq=3.0), product of:
          0.3084192 = queryWeight, product of:
            3.673553 = boost
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.016636252 = queryNorm
          0.5463113 = fieldWeight in 636, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.0625 = fieldNorm(doc=636)
    0.36 = coord(9/25)
  3. Kantor, P.; Kim, M.H.; Ibraev, U.; Atasoy, K.: Estimating the number of relevant documents in enormous collections (1999) 0.32 
    0.32433298 = sum of:
  0.32433298 = product of:
    0.9009249 = sum of:
      0.03949022 = weight(abstract_txt:documents in 6690) [ClassicSimilarity], result of:
        0.03949022 = score(doc=6690,freq=5.0), product of:
          0.06856303 = queryWeight, product of:
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.016636252 = queryNorm
          0.5759696 = fieldWeight in 6690, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.07539228 = weight(abstract_txt:relevant in 6690) [ClassicSimilarity], result of:
        0.07539228 = score(doc=6690,freq=9.0), product of:
          0.08674093 = queryWeight, product of:
            1.1247786 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.016636252 = queryNorm
          0.86916614 = fieldWeight in 6690, product of:
            3.0 = tf(freq=9.0), with freq of:
              9.0 = termFreq=9.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.09281744 = weight(abstract_txt:retrieving in 6690) [ClassicSimilarity], result of:
        0.09281744 = score(doc=6690,freq=2.0), product of:
          0.16449904 = queryWeight, product of:
            1.5489475 = boost
            6.3836813 = idf(docFreq=202, maxDocs=44218)
            0.016636252 = queryNorm
          0.564243 = fieldWeight in 6690, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            6.3836813 = idf(docFreq=202, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.037434556 = weight(abstract_txt:retrieval in 6690) [ClassicSimilarity], result of:
        0.037434556 = score(doc=6690,freq=2.0), product of:
          0.12187243 = queryWeight, product of:
            2.1080346 = boost
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.016636252 = queryNorm
          0.3071618 = fieldWeight in 6690, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.19711249 = weight(abstract_txt:trec in 6690) [ClassicSimilarity], result of:
        0.19711249 = score(doc=6690,freq=3.0), product of:
          0.2717814 = queryWeight, product of:
            2.4384313 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.016636252 = queryNorm
          0.7252611 = fieldWeight in 6690, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.07514213 = weight(abstract_txt:performance in 6690) [ClassicSimilarity], result of:
        0.07514213 = score(doc=6690,freq=1.0), product of:
          0.25964686 = queryWeight, product of:
            3.3705995 = boost
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.016636252 = queryNorm
          0.28940126 = fieldWeight in 6690, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.63042 = idf(docFreq=1171, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.17556418 = weight(abstract_txt:chance in 6690) [ClassicSimilarity], result of:
        0.17556418 = score(doc=6690,freq=1.0), product of:
          0.3993812 = queryWeight, product of:
            3.413216 = boost
            7.033448 = idf(docFreq=105, maxDocs=44218)
            0.016636252 = queryNorm
          0.4395905 = fieldWeight in 6690, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.033448 = idf(docFreq=105, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.11069221 = weight(abstract_txt:collections in 6690) [ClassicSimilarity], result of:
        0.11069221 = score(doc=6690,freq=2.0), product of:
          0.26680592 = queryWeight, product of:
            3.4167511 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.016636252 = queryNorm
          0.41487914 = fieldWeight in 6690, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
      0.09727942 = weight(abstract_txt:test in 6690) [ClassicSimilarity], result of:
        0.09727942 = score(doc=6690,freq=1.0), product of:
          0.3084192 = queryWeight, product of:
            3.673553 = boost
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.016636252 = queryNorm
          0.315413 = fieldWeight in 6690, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.0625 = fieldNorm(doc=6690)
    0.36 = coord(9/25)
  4. Fujita, S.: Technology survey and invalidity search : a comparative study of different tasks for Japanese patent document retrieval (2007) 0.24 
    0.24484242 = sum of:
  0.24484242 = product of:
    0.7651326 = sum of:
      0.03746371 = weight(abstract_txt:documents in 918) [ClassicSimilarity], result of:
        0.03746371 = score(doc=918,freq=2.0), product of:
          0.06856303 = queryWeight, product of:
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.016636252 = queryNorm
          0.5464127 = fieldWeight in 918, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.053310394 = weight(abstract_txt:relevant in 918) [ClassicSimilarity], result of:
        0.053310394 = score(doc=918,freq=2.0), product of:
          0.08674093 = queryWeight, product of:
            1.1247786 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.016636252 = queryNorm
          0.61459327 = fieldWeight in 918, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.053518414 = weight(abstract_txt:models in 918) [ClassicSimilarity], result of:
        0.053518414 = score(doc=918,freq=2.0), product of:
          0.08696643 = queryWeight, product of:
            1.1262397 = boost
            4.6415744 = idf(docFreq=1158, maxDocs=44218)
            0.016636252 = queryNorm
          0.6153916 = fieldWeight in 918, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.6415744 = idf(docFreq=1158, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.07941069 = weight(abstract_txt:retrieval in 918) [ClassicSimilarity], result of:
        0.07941069 = score(doc=918,freq=4.0), product of:
          0.12187243 = queryWeight, product of:
            2.1080346 = boost
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.016636252 = queryNorm
          0.6515886 = fieldWeight in 918, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.10078925 = weight(abstract_txt:queries in 918) [ClassicSimilarity], result of:
        0.10078925 = score(doc=918,freq=1.0), product of:
          0.21052918 = queryWeight, product of:
            2.4781418 = boost
            5.106586 = idf(docFreq=727, maxDocs=44218)
            0.016636252 = queryNorm
          0.47874242 = fieldWeight in 918, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.106586 = idf(docFreq=727, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.17731415 = weight(abstract_txt:effectiveness in 918) [ClassicSimilarity], result of:
        0.17731415 = score(doc=918,freq=2.0), product of:
          0.26231617 = queryWeight, product of:
            3.092698 = boost
            5.098378 = idf(docFreq=733, maxDocs=44218)
            0.016636252 = queryNorm
          0.67595583 = fieldWeight in 918, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.098378 = idf(docFreq=733, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.11740682 = weight(abstract_txt:collections in 918) [ClassicSimilarity], result of:
        0.11740682 = score(doc=918,freq=1.0), product of:
          0.26680592 = queryWeight, product of:
            3.4167511 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.016636252 = queryNorm
          0.4400458 = fieldWeight in 918, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
      0.14591913 = weight(abstract_txt:test in 918) [ClassicSimilarity], result of:
        0.14591913 = score(doc=918,freq=1.0), product of:
          0.3084192 = queryWeight, product of:
            3.673553 = boost
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.016636252 = queryNorm
          0.4731195 = fieldWeight in 918, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.09375 = fieldNorm(doc=918)
    0.32 = coord(8/25)
  5. Voorhees, E.M.; Harman, D.K.: ¬The Text REtrieval Conference (2005) 0.24 
    0.23643735 = sum of:
  0.23643735 = product of:
    0.6567704 = sum of:
      0.011037852 = weight(abstract_txt:documents in 5082) [ClassicSimilarity], result of:
        0.011037852 = score(doc=5082,freq=1.0), product of:
          0.06856303 = queryWeight, product of:
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.016636252 = queryNorm
          0.16098839 = fieldWeight in 5082, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.015706725 = weight(abstract_txt:relevant in 5082) [ClassicSimilarity], result of:
        0.015706725 = score(doc=5082,freq=1.0), product of:
          0.08674093 = queryWeight, product of:
            1.1247786 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.016636252 = queryNorm
          0.18107629 = fieldWeight in 5082, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.02385172 = weight(abstract_txt:traditional in 5082) [ClassicSimilarity], result of:
        0.02385172 = score(doc=5082,freq=1.0), product of:
          0.1311829 = queryWeight, product of:
            1.6941007 = boost
            4.654601 = idf(docFreq=1143, maxDocs=44218)
            0.016636252 = queryNorm
          0.18182036 = fieldWeight in 5082, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.654601 = idf(docFreq=1143, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.06190158 = weight(abstract_txt:retrieval in 5082) [ClassicSimilarity], result of:
        0.06190158 = score(doc=5082,freq=14.0), product of:
          0.12187243 = queryWeight, product of:
            2.1080346 = boost
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.016636252 = queryNorm
          0.5079211 = fieldWeight in 5082, product of:
            3.7416575 = tf(freq=14.0), with freq of:
              14.0 = termFreq=14.0
            3.4751394 = idf(docFreq=3720, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.2932635 = weight(abstract_txt:trec in 5082) [ClassicSimilarity], result of:
        0.2932635 = score(doc=5082,freq=17.0), product of:
          0.2717814 = queryWeight, product of:
            2.4384313 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.016636252 = queryNorm
          1.0790417 = fieldWeight in 5082, product of:
            4.1231055 = tf(freq=17.0), with freq of:
              17.0 = termFreq=17.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.04360107 = weight(abstract_txt:standards in 5082) [ClassicSimilarity], result of:
        0.04360107 = score(doc=5082,freq=1.0), product of:
          0.23252968 = queryWeight, product of:
            2.9118178 = boost
            4.800193 = idf(docFreq=988, maxDocs=44218)
            0.016636252 = queryNorm
          0.18750754 = fieldWeight in 5082, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.800193 = idf(docFreq=988, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.05224168 = weight(abstract_txt:effectiveness in 5082) [ClassicSimilarity], result of:
        0.05224168 = score(doc=5082,freq=1.0), product of:
          0.26231617 = queryWeight, product of:
            3.092698 = boost
            5.098378 = idf(docFreq=733, maxDocs=44218)
            0.016636252 = queryNorm
          0.19915539 = fieldWeight in 5082, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.098378 = idf(docFreq=733, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.06918263 = weight(abstract_txt:collections in 5082) [ClassicSimilarity], result of:
        0.06918263 = score(doc=5082,freq=2.0), product of:
          0.26680592 = queryWeight, product of:
            3.4167511 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.016636252 = queryNorm
          0.25929946 = fieldWeight in 5082, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
      0.08598367 = weight(abstract_txt:test in 5082) [ClassicSimilarity], result of:
        0.08598367 = score(doc=5082,freq=2.0), product of:
          0.3084192 = queryWeight, product of:
            3.673553 = boost
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.016636252 = queryNorm
          0.27878833 = fieldWeight in 5082, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.046608 = idf(docFreq=772, maxDocs=44218)
            0.0390625 = fieldNorm(doc=5082)
    0.36 = coord(9/25)