Document (#24691)

Author
Kantor, P.
Kim, M.H.
Ibraev, U.
Atasoy, K.
Title
Estimating the number of relevant documents in enormous collections
Source
Knowledge: creation, organization and use. Proceedings of the 62nd Annual Meeting of the American Society for Information Science, 31.10.-4.11.1999. Ed.: L. Woods
Imprint
Medford, NJ : Information Today
Year
1999
Pages
S.507-514
Series
Proceedings of the American Society for Information Science; vol.36
Abstract
In assessing information retrieval systems, it is important to know not only the precision of the retrieved set, but also to compare the number of retrieved relevant items to the total number of relevant items. For large collections, such as the TREC test collections, or the World Wide Web, it is not possible to enumerate the entire set of relevant documents. If the retrieved documents are evaluated, a variant of the statistical "capture-recapture" method can be used to estimate the total number of relevant documents, providing the several retrieval systems used are sufficiently independent. We show that the underlying signal detection model supporting such an analysis can be extended in two ways. First, assuming that there are two distinct performance characteristics (corresponding to the chance of retrieving a relevant, and retrieving a given non-relevant document), we show that if there are three or more independent systems available it is possible to estimate the number of relevant documents without actually having to decide whether each individual document is relevant. We report applications of this 3-system method to the TREC data, leading to the conclusion that the independence assumptions are not satisfied. We then extend the model to a multi-system, multi-problem model, and show that it is possible to include statistical dependencies of all orders in the model, and determine the number of relevant documents for each of the problems in the set. Application to the TREC setting will be presented
Theme
Retrievalalgorithmen
Retrievalstudien
Object
TREC

Similar documents (author)

  1. Kantor, P.B.: ¬The Adaptive Network Library Interface : a historical overview and interim report (1993) 5.11 
    5.106579 = sum of:
  5.106579 = weight(author_txt:kantor in 6976) [ClassicSimilarity], result of:
    5.106579 = fieldWeight in 6976, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.1705265 = idf(docFreq=33, maxDocs=44218)
      0.625 = fieldNorm(doc=6976)
  2. Kantor, P.B.: ¬The logic of weighted queries (1981) 5.11 
    5.106579 = sum of:
  5.106579 = weight(author_txt:kantor in 7622) [ClassicSimilarity], result of:
    5.106579 = fieldWeight in 7622, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.1705265 = idf(docFreq=33, maxDocs=44218)
      0.625 = fieldNorm(doc=7622)
  3. Kantor, A.: Baby steps (1994) 5.11 
    5.106579 = sum of:
  5.106579 = weight(author_txt:kantor in 7985) [ClassicSimilarity], result of:
    5.106579 = fieldWeight in 7985, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.1705265 = idf(docFreq=33, maxDocs=44218)
      0.625 = fieldNorm(doc=7985)
  4. Kantor, P.B.: Information retrieval techniques (1994) 5.11 
    5.106579 = sum of:
  5.106579 = weight(author_txt:kantor in 1056) [ClassicSimilarity], result of:
    5.106579 = fieldWeight in 1056, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.1705265 = idf(docFreq=33, maxDocs=44218)
      0.625 = fieldNorm(doc=1056)
  5. Kantor, P.B.: ¬A model for stopping behavior of the users of on-line systems (1987) 5.11 
    5.106579 = sum of:
  5.106579 = weight(author_txt:kantor in 3945) [ClassicSimilarity], result of:
    5.106579 = fieldWeight in 3945, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      8.1705265 = idf(docFreq=33, maxDocs=44218)
      0.625 = fieldNorm(doc=3945)

Similar documents (content)

  1. Collins-Thompson, K.; Callan, J.: Predicting reading difficulty with statistical language models (2005) 0.40 
    0.4015719 = sum of:
  0.4015719 = product of:
    0.7722537 = sum of:
      0.020949796 = weight(abstract_txt:each in 4579) [ClassicSimilarity], result of:
        0.020949796 = score(doc=4579,freq=1.0), product of:
          0.08138328 = queryWeight, product of:
            1.0286558 = boost
            4.118742 = idf(docFreq=1954, maxDocs=44218)
            0.019208813 = queryNorm
          0.25742137 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.118742 = idf(docFreq=1954, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.023716459 = weight(abstract_txt:document in 4579) [ClassicSimilarity], result of:
        0.023716459 = score(doc=4579,freq=1.0), product of:
          0.08839925 = queryWeight, product of:
            1.072079 = boost
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.019208813 = queryNorm
          0.26828802 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.027340002 = weight(abstract_txt:method in 4579) [ClassicSimilarity], result of:
        0.027340002 = score(doc=4579,freq=1.0), product of:
          0.09718838 = queryWeight, product of:
            1.1241122 = boost
            4.50095 = idf(docFreq=1333, maxDocs=44218)
            0.019208813 = queryNorm
          0.28130937 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.50095 = idf(docFreq=1333, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.017863143 = weight(abstract_txt:systems in 4579) [ClassicSimilarity], result of:
        0.017863143 = score(doc=4579,freq=1.0), product of:
          0.083769195 = queryWeight, product of:
            1.2781749 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.019208813 = queryNorm
          0.2132424 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.07234483 = weight(abstract_txt:statistical in 4579) [ClassicSimilarity], result of:
        0.07234483 = score(doc=4579,freq=2.0), product of:
          0.14757417 = queryWeight, product of:
            1.3851846 = boost
            5.5462847 = idf(docFreq=468, maxDocs=44218)
            0.019208813 = queryNorm
          0.49022692 = fieldWeight in 4579, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.5462847 = idf(docFreq=468, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.022298083 = weight(abstract_txt:that in 4579) [ClassicSimilarity], result of:
        0.022298083 = score(doc=4579,freq=5.0), product of:
          0.06733647 = queryWeight, product of:
            1.479441 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.019208813 = queryNorm
          0.3311442 = fieldWeight in 4579, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.05440012 = weight(abstract_txt:show in 4579) [ClassicSimilarity], result of:
        0.05440012 = score(doc=4579,freq=2.0), product of:
          0.13969119 = queryWeight, product of:
            1.6505649 = boost
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.019208813 = queryNorm
          0.3894313 = fieldWeight in 4579, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.06293843 = weight(abstract_txt:possible in 4579) [ClassicSimilarity], result of:
        0.06293843 = score(doc=4579,freq=2.0), product of:
          0.15394999 = queryWeight, product of:
            1.7327582 = boost
            4.6253138 = idf(docFreq=1177, maxDocs=44218)
            0.019208813 = queryNorm
          0.40882385 = fieldWeight in 4579, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.6253138 = idf(docFreq=1177, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.04651116 = weight(abstract_txt:collections in 4579) [ClassicSimilarity], result of:
        0.04651116 = score(doc=4579,freq=1.0), product of:
          0.15854426 = queryWeight, product of:
            1.7584231 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.019208813 = queryNorm
          0.29336387 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.11177757 = weight(abstract_txt:estimate in 4579) [ClassicSimilarity], result of:
        0.11177757 = score(doc=4579,freq=1.0), product of:
          0.24849544 = queryWeight, product of:
            1.7974702 = boost
            7.1970778 = idf(docFreq=89, maxDocs=44218)
            0.019208813 = queryNorm
          0.44981736 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.1970778 = idf(docFreq=89, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.05371794 = weight(abstract_txt:model in 4579) [ClassicSimilarity], result of:
        0.05371794 = score(doc=4579,freq=2.0), product of:
          0.15246199 = queryWeight, product of:
            1.9911239 = boost
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.019208813 = queryNorm
          0.35233662 = fieldWeight in 4579, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.109061494 = weight(abstract_txt:documents in 4579) [ClassicSimilarity], result of:
        0.109061494 = score(doc=4579,freq=3.0), product of:
          0.24445353 = queryWeight, product of:
            3.087886 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.019208813 = queryNorm
          0.44614407 = fieldWeight in 4579, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
      0.14933468 = weight(abstract_txt:relevant in 4579) [ClassicSimilarity], result of:
        0.14933468 = score(doc=4579,freq=1.0), product of:
          0.5154412 = queryWeight, product of:
            5.7886467 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.019208813 = queryNorm
          0.28972206 = fieldWeight in 4579, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=4579)
    0.52 = coord(13/25)
  2. Keenan, S.; Smeaton, A.F.; Keogh, G.: ¬The effect of pool depth on system evaluation in TREC (2001) 0.38 
    0.3828976 = sum of:
  0.3828976 = product of:
    0.9572439 = sum of:
      0.020657413 = weight(abstract_txt:there in 5908) [ClassicSimilarity], result of:
        0.020657413 = score(doc=5908,freq=1.0), product of:
          0.0806243 = queryWeight, product of:
            1.0238479 = boost
            4.099491 = idf(docFreq=1992, maxDocs=44218)
            0.019208813 = queryNorm
          0.2562182 = fieldWeight in 5908, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.099491 = idf(docFreq=1992, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.03628611 = weight(abstract_txt:each in 5908) [ClassicSimilarity], result of:
        0.03628611 = score(doc=5908,freq=3.0), product of:
          0.08138328 = queryWeight, product of:
            1.0286558 = boost
            4.118742 = idf(docFreq=1954, maxDocs=44218)
            0.019208813 = queryNorm
          0.44586688 = fieldWeight in 5908, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.118742 = idf(docFreq=1954, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.030939871 = weight(abstract_txt:systems in 5908) [ClassicSimilarity], result of:
        0.030939871 = score(doc=5908,freq=3.0), product of:
          0.083769195 = queryWeight, product of:
            1.2781749 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.019208813 = queryNorm
          0.36934665 = fieldWeight in 5908, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.014102546 = weight(abstract_txt:that in 5908) [ClassicSimilarity], result of:
        0.014102546 = score(doc=5908,freq=2.0), product of:
          0.06733647 = queryWeight, product of:
            1.479441 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.019208813 = queryNorm
          0.20943399 = fieldWeight in 5908, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.037984323 = weight(abstract_txt:model in 5908) [ClassicSimilarity], result of:
        0.037984323 = score(doc=5908,freq=1.0), product of:
          0.15246199 = queryWeight, product of:
            1.9911239 = boost
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.019208813 = queryNorm
          0.24913962 = fieldWeight in 5908, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.08363966 = weight(abstract_txt:retrieved in 5908) [ClassicSimilarity], result of:
        0.08363966 = score(doc=5908,freq=1.0), product of:
          0.234452 = queryWeight, product of:
            2.1383317 = boost
            5.707926 = idf(docFreq=398, maxDocs=44218)
            0.019208813 = queryNorm
          0.35674536 = fieldWeight in 5908, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.707926 = idf(docFreq=398, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.30242905 = weight(abstract_txt:trec in 5908) [ClassicSimilarity], result of:
        0.30242905 = score(doc=5908,freq=5.0), product of:
          0.32300165 = queryWeight, product of:
            2.5098658 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.019208813 = queryNorm
          0.9363081 = fieldWeight in 5908, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.109061494 = weight(abstract_txt:documents in 5908) [ClassicSimilarity], result of:
        0.109061494 = score(doc=5908,freq=3.0), product of:
          0.24445353 = queryWeight, product of:
            3.087886 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.019208813 = queryNorm
          0.44614407 = fieldWeight in 5908, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.06348818 = weight(abstract_txt:number in 5908) [ClassicSimilarity], result of:
        0.06348818 = score(doc=5908,freq=1.0), product of:
          0.24580139 = queryWeight, product of:
            3.0963874 = boost
            4.132649 = idf(docFreq=1927, maxDocs=44218)
            0.019208813 = queryNorm
          0.25829056 = fieldWeight in 5908, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.132649 = idf(docFreq=1927, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
      0.25865525 = weight(abstract_txt:relevant in 5908) [ClassicSimilarity], result of:
        0.25865525 = score(doc=5908,freq=3.0), product of:
          0.5154412 = queryWeight, product of:
            5.7886467 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.019208813 = queryNorm
          0.5018133 = fieldWeight in 5908, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=5908)
    0.4 = coord(10/25)
  3. Liu, X.; Zheng, W.; Fang, H.: ¬An exploration of ranking models and feedback method for related entity finding (2013) 0.35 
    0.34889457 = sum of:
  0.34889457 = product of:
    0.7929422 = sum of:
      0.023716459 = weight(abstract_txt:document in 2714) [ClassicSimilarity], result of:
        0.023716459 = score(doc=2714,freq=1.0), product of:
          0.08839925 = queryWeight, product of:
            1.072079 = boost
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.019208813 = queryNorm
          0.26828802 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.0386646 = weight(abstract_txt:method in 2714) [ClassicSimilarity], result of:
        0.0386646 = score(doc=2714,freq=2.0), product of:
          0.09718838 = queryWeight, product of:
            1.1241122 = boost
            4.50095 = idf(docFreq=1333, maxDocs=44218)
            0.019208813 = queryNorm
          0.3978315 = fieldWeight in 2714, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.50095 = idf(docFreq=1333, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.014102546 = weight(abstract_txt:that in 2714) [ClassicSimilarity], result of:
        0.014102546 = score(doc=2714,freq=2.0), product of:
          0.06733647 = queryWeight, product of:
            1.479441 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.019208813 = queryNorm
          0.20943399 = fieldWeight in 2714, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.038466696 = weight(abstract_txt:show in 2714) [ClassicSimilarity], result of:
        0.038466696 = score(doc=2714,freq=1.0), product of:
          0.13969119 = queryWeight, product of:
            1.6505649 = boost
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.019208813 = queryNorm
          0.27536952 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.044504188 = weight(abstract_txt:possible in 2714) [ClassicSimilarity], result of:
        0.044504188 = score(doc=2714,freq=1.0), product of:
          0.15394999 = queryWeight, product of:
            1.7327582 = boost
            4.6253138 = idf(docFreq=1177, maxDocs=44218)
            0.019208813 = queryNorm
          0.2890821 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.6253138 = idf(docFreq=1177, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.04651116 = weight(abstract_txt:collections in 2714) [ClassicSimilarity], result of:
        0.04651116 = score(doc=2714,freq=1.0), product of:
          0.15854426 = queryWeight, product of:
            1.7584231 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.019208813 = queryNorm
          0.29336387 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.11177757 = weight(abstract_txt:estimate in 2714) [ClassicSimilarity], result of:
        0.11177757 = score(doc=2714,freq=1.0), product of:
          0.24849544 = queryWeight, product of:
            1.7974702 = boost
            7.1970778 = idf(docFreq=89, maxDocs=44218)
            0.019208813 = queryNorm
          0.44981736 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.1970778 = idf(docFreq=89, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.06579078 = weight(abstract_txt:model in 2714) [ClassicSimilarity], result of:
        0.06579078 = score(doc=2714,freq=3.0), product of:
          0.15246199 = queryWeight, product of:
            1.9911239 = boost
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.019208813 = queryNorm
          0.4315225 = fieldWeight in 2714, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.13525039 = weight(abstract_txt:trec in 2714) [ClassicSimilarity], result of:
        0.13525039 = score(doc=2714,freq=1.0), product of:
          0.32300165 = queryWeight, product of:
            2.5098658 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.019208813 = queryNorm
          0.4187297 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.06296669 = weight(abstract_txt:documents in 2714) [ClassicSimilarity], result of:
        0.06296669 = score(doc=2714,freq=1.0), product of:
          0.24445353 = queryWeight, product of:
            3.087886 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.019208813 = queryNorm
          0.2575814 = fieldWeight in 2714, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
      0.21119112 = weight(abstract_txt:relevant in 2714) [ClassicSimilarity], result of:
        0.21119112 = score(doc=2714,freq=2.0), product of:
          0.5154412 = queryWeight, product of:
            5.7886467 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.019208813 = queryNorm
          0.40972885 = fieldWeight in 2714, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=2714)
    0.44 = coord(11/25)
  4. Dang, E.K.F.; Luk, R.W.P.; Allan, J.: ¬A context-dependent relevance model (2016) 0.35 
    0.34612656 = sum of:
  0.34612656 = product of:
    0.78665125 = sum of:
      0.08006812 = weight(abstract_txt:estimating in 2778) [ClassicSimilarity], result of:
        0.08006812 = score(doc=2778,freq=1.0), product of:
          0.15789865 = queryWeight, product of:
            1.0131569 = boost
            8.113368 = idf(docFreq=35, maxDocs=44218)
            0.019208813 = queryNorm
          0.5070855 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.113368 = idf(docFreq=35, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.033540137 = weight(abstract_txt:document in 2778) [ClassicSimilarity], result of:
        0.033540137 = score(doc=2778,freq=2.0), product of:
          0.08839925 = queryWeight, product of:
            1.072079 = boost
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.019208813 = queryNorm
          0.37941656 = fieldWeight in 2778, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.0386646 = weight(abstract_txt:method in 2778) [ClassicSimilarity], result of:
        0.0386646 = score(doc=2778,freq=2.0), product of:
          0.09718838 = queryWeight, product of:
            1.1241122 = boost
            4.50095 = idf(docFreq=1333, maxDocs=44218)
            0.019208813 = queryNorm
          0.3978315 = fieldWeight in 2778, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.50095 = idf(docFreq=1333, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.014102546 = weight(abstract_txt:that in 2778) [ClassicSimilarity], result of:
        0.014102546 = score(doc=2778,freq=2.0), product of:
          0.06733647 = queryWeight, product of:
            1.479441 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.019208813 = queryNorm
          0.20943399 = fieldWeight in 2778, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.038466696 = weight(abstract_txt:show in 2778) [ClassicSimilarity], result of:
        0.038466696 = score(doc=2778,freq=1.0), product of:
          0.13969119 = queryWeight, product of:
            1.6505649 = boost
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.019208813 = queryNorm
          0.27536952 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.4059124 = idf(docFreq=1466, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.04651116 = weight(abstract_txt:collections in 2778) [ClassicSimilarity], result of:
        0.04651116 = score(doc=2778,freq=1.0), product of:
          0.15854426 = queryWeight, product of:
            1.7584231 = boost
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.019208813 = queryNorm
          0.29336387 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.693822 = idf(docFreq=1099, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.11177757 = weight(abstract_txt:estimate in 2778) [ClassicSimilarity], result of:
        0.11177757 = score(doc=2778,freq=1.0), product of:
          0.24849544 = queryWeight, product of:
            1.7974702 = boost
            7.1970778 = idf(docFreq=89, maxDocs=44218)
            0.019208813 = queryNorm
          0.44981736 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.1970778 = idf(docFreq=89, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.075968646 = weight(abstract_txt:model in 2778) [ClassicSimilarity], result of:
        0.075968646 = score(doc=2778,freq=4.0), product of:
          0.15246199 = queryWeight, product of:
            1.9911239 = boost
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.019208813 = queryNorm
          0.49827924 = fieldWeight in 2778, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.986234 = idf(docFreq=2231, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.13525039 = weight(abstract_txt:trec in 2778) [ClassicSimilarity], result of:
        0.13525039 = score(doc=2778,freq=1.0), product of:
          0.32300165 = queryWeight, product of:
            2.5098658 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.019208813 = queryNorm
          0.4187297 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.06296669 = weight(abstract_txt:documents in 2778) [ClassicSimilarity], result of:
        0.06296669 = score(doc=2778,freq=1.0), product of:
          0.24445353 = queryWeight, product of:
            3.087886 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.019208813 = queryNorm
          0.2575814 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
      0.14933468 = weight(abstract_txt:relevant in 2778) [ClassicSimilarity], result of:
        0.14933468 = score(doc=2778,freq=1.0), product of:
          0.5154412 = queryWeight, product of:
            5.7886467 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.019208813 = queryNorm
          0.28972206 = fieldWeight in 2778, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=2778)
    0.44 = coord(11/25)
  5. Vakkari, P.; Jones, S.; MacFarlane, A.; Sormunen, E.: Query exhaustivity, relevance feedback and search success in automatic and interactive query expansion (2004) 0.32 
    0.320435 = sum of:
  0.320435 = product of:
    0.89009714 = sum of:
      0.020657413 = weight(abstract_txt:there in 4435) [ClassicSimilarity], result of:
        0.020657413 = score(doc=4435,freq=1.0), product of:
          0.0806243 = queryWeight, product of:
            1.0238479 = boost
            4.099491 = idf(docFreq=1992, maxDocs=44218)
            0.019208813 = queryNorm
          0.2562182 = fieldWeight in 4435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.099491 = idf(docFreq=1992, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.020949796 = weight(abstract_txt:each in 4435) [ClassicSimilarity], result of:
        0.020949796 = score(doc=4435,freq=1.0), product of:
          0.08138328 = queryWeight, product of:
            1.0286558 = boost
            4.118742 = idf(docFreq=1954, maxDocs=44218)
            0.019208813 = queryNorm
          0.25742137 = fieldWeight in 4435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.118742 = idf(docFreq=1954, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.07362429 = weight(abstract_txt:items in 4435) [ClassicSimilarity], result of:
        0.07362429 = score(doc=4435,freq=2.0), product of:
          0.14930902 = queryWeight, product of:
            1.3933028 = boost
            5.57879 = idf(docFreq=453, maxDocs=44218)
            0.019208813 = queryNorm
          0.49310005 = fieldWeight in 4435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.57879 = idf(docFreq=453, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.014102546 = weight(abstract_txt:that in 4435) [ClassicSimilarity], result of:
        0.014102546 = score(doc=4435,freq=2.0), product of:
          0.06733647 = queryWeight, product of:
            1.479441 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.019208813 = queryNorm
          0.20943399 = fieldWeight in 4435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.11828434 = weight(abstract_txt:retrieved in 4435) [ClassicSimilarity], result of:
        0.11828434 = score(doc=4435,freq=2.0), product of:
          0.234452 = queryWeight, product of:
            2.1383317 = boost
            5.707926 = idf(docFreq=398, maxDocs=44218)
            0.019208813 = queryNorm
          0.5045141 = fieldWeight in 4435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.707926 = idf(docFreq=398, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.19127293 = weight(abstract_txt:trec in 4435) [ClassicSimilarity], result of:
        0.19127293 = score(doc=4435,freq=2.0), product of:
          0.32300165 = queryWeight, product of:
            2.5098658 = boost
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.019208813 = queryNorm
          0.5921732 = fieldWeight in 4435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            6.699675 = idf(docFreq=147, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.08904834 = weight(abstract_txt:documents in 4435) [ClassicSimilarity], result of:
        0.08904834 = score(doc=4435,freq=2.0), product of:
          0.24445353 = queryWeight, product of:
            3.087886 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.019208813 = queryNorm
          0.36427513 = fieldWeight in 4435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.06348818 = weight(abstract_txt:number in 4435) [ClassicSimilarity], result of:
        0.06348818 = score(doc=4435,freq=1.0), product of:
          0.24580139 = queryWeight, product of:
            3.0963874 = boost
            4.132649 = idf(docFreq=1927, maxDocs=44218)
            0.019208813 = queryNorm
          0.25829056 = fieldWeight in 4435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.132649 = idf(docFreq=1927, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
      0.29866937 = weight(abstract_txt:relevant in 4435) [ClassicSimilarity], result of:
        0.29866937 = score(doc=4435,freq=4.0), product of:
          0.5154412 = queryWeight, product of:
            5.7886467 = boost
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.019208813 = queryNorm
          0.5794441 = fieldWeight in 4435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.635553 = idf(docFreq=1165, maxDocs=44218)
            0.0625 = fieldNorm(doc=4435)
    0.36 = coord(9/25)