Document (#23909)

Author

Keenan, S.
Smeaton, A.F.
Keogh, G.

Title

¬The effect of pool depth on system evaluation in TREC

Source

Journal of the American Society for Information Science and technology. 52(2001) no.7, S.570-574

Year

2001

Abstract

The TREC benchmarking exercise for information retrieval (IR) experiments has provided a forum and an opportunity for IR researchers to evaluate the performance of their approaches to the IR task and has resulted in improvements in IR effectiveness. Typically, retrieval performance has been measured in terms of precision and recall, and comparisons between different IR approaches have been based on these measures. These measures are in turn dependent on the so-called "pool depth" used to discover relevant documents. Whereas there is evidence to suggest that the pool depth size used for TREC evaluations adequately identifies the relevant documents in the entire test data collection, we consider how it affects the evaluations of individual systems. The data used comes from the Sixth TREC conference, TREC-6. By fitting appropriate regression models we explore whether different pool depths confer advantages or disadvantages on different retrieval systems when they are compared. As a consequence of this model fitting, a pair of measures for each retrieval run, which are related to precision and recall, emerge. For each system, these give an extrapolation for the number of relevant documents the system would have been deemed to have retrieved if an indefinitely large pool size had been used, and also a measure of the sensitivity of each system to pool size. We concur that even on the basis of analyses of individual systems, the pool depth of 100 used by TREC is adequate

Theme

Retrievalstudien

Object

TREC

Similar documents (author)

1. Keenan, S.: Concise dictionary of library and information science (1996) 2.29

   2.2947989 = sum of:
     2.2947989 = product of:
       4.5895977 = sum of:
         4.5895977 = weight(author_txt:keenan in 3437) [ClassicSimilarity], result of:
           4.5895977 = score(doc=3437,freq=1.0), product of:
             0.7730123 = queryWeight, product of:
               1.1038617 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.073716305 = queryNorm
             5.937289 = fieldWeight in 3437, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.625 = fieldNorm(doc=3437)
       0.5 = coord(1/2)
   

2. Keenan, S.: From world bibliography to the Internet in 100 years : FID celebrates its centennial (1995) 2.29

   2.2947989 = sum of:
     2.2947989 = product of:
       4.5895977 = sum of:
         4.5895977 = weight(author_txt:keenan in 1148) [ClassicSimilarity], result of:
           4.5895977 = score(doc=1148,freq=1.0), product of:
             0.7730123 = queryWeight, product of:
               1.1038617 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.073716305 = queryNorm
             5.937289 = fieldWeight in 1148, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.625 = fieldNorm(doc=1148)
       0.5 = coord(1/2)
   

3. Keenan, S.: Abstracting and indexing services in science and technology (1969) 2.29

   2.2947989 = sum of:
     2.2947989 = product of:
       4.5895977 = sum of:
         4.5895977 = weight(author_txt:keenan in 235) [ClassicSimilarity], result of:
           4.5895977 = score(doc=235,freq=1.0), product of:
             0.7730123 = queryWeight, product of:
               1.1038617 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.073716305 = queryNorm
             5.937289 = fieldWeight in 235, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.625 = fieldNorm(doc=235)
       0.5 = coord(1/2)
   

4. Keenan, J.P.: ¬Das Gesicht im Spiegel : Auf der Suche nach dem Ursprung des Bewusstseins (2005) 2.29

   2.2947989 = sum of:
     2.2947989 = product of:
       4.5895977 = sum of:
         4.5895977 = weight(author_txt:keenan in 4844) [ClassicSimilarity], result of:
           4.5895977 = score(doc=4844,freq=1.0), product of:
             0.7730123 = queryWeight, product of:
               1.1038617 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.073716305 = queryNorm
             5.937289 = fieldWeight in 4844, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.625 = fieldNorm(doc=4844)
       0.5 = coord(1/2)
   

5. Keenan, S.; Johnston, C.: Concise dictionary of library and information science (2000) 1.84

   1.8358389 = sum of:
     1.8358389 = product of:
       3.6716778 = sum of:
         3.6716778 = weight(author_txt:keenan in 1354) [ClassicSimilarity], result of:
           3.6716778 = score(doc=1354,freq=1.0), product of:
             0.7730123 = queryWeight, product of:
               1.1038617 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.073716305 = queryNorm
             4.749831 = fieldWeight in 1354, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.5 = fieldNorm(doc=1354)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Pal, S.; Mitra, M.; Kamps, J.: Evaluation effort, reliability and reusability in XML retrieval (2011) 0.44

   0.4398165 = sum of:
     0.4398165 = product of:
       0.9162844 = sum of:
         0.008450585 = weight(abstract_txt:these in 4197) [ClassicSimilarity], result of:
           0.008450585 = score(doc=4197,freq=1.0), product of:
             0.048468858 = queryWeight, product of:
               1.0927672 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.013912308 = queryNorm
             0.17435083 = fieldWeight in 4197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.020715222 = weight(abstract_txt:systems in 4197) [ClassicSimilarity], result of:
           0.020715222 = score(doc=4197,freq=4.0), product of:
             0.055510864 = queryWeight, product of:
               1.1694597 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.013912308 = queryNorm
             0.3731742 = fieldWeight in 4197, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.050791282 = weight(abstract_txt:precision in 4197) [ClassicSimilarity], result of:
           0.050791282 = score(doc=4197,freq=3.0), product of:
             0.09704955 = queryWeight, product of:
               1.2625465 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.013912308 = queryNorm
             0.5233541 = fieldWeight in 4197, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.046713904 = weight(abstract_txt:recall in 4197) [ClassicSimilarity], result of:
           0.046713904 = score(doc=4197,freq=2.0), product of:
             0.105065934 = queryWeight, product of:
               1.3136559 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.013912308 = queryNorm
             0.44461513 = fieldWeight in 4197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.013335202 = weight(abstract_txt:system in 4197) [ClassicSimilarity], result of:
           0.013335202 = score(doc=4197,freq=1.0), product of:
             0.072307646 = queryWeight, product of:
               1.5411955 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.013912308 = queryNorm
             0.18442312 = fieldWeight in 4197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.029185316 = weight(abstract_txt:retrieval in 4197) [ClassicSimilarity], result of:
           0.029185316 = score(doc=4197,freq=4.0), product of:
             0.076784596 = queryWeight, product of:
               1.5881908 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013912308 = queryNorm
             0.38009337 = fieldWeight in 4197, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.01646159 = weight(abstract_txt:been in 4197) [ClassicSimilarity], result of:
           0.01646159 = score(doc=4197,freq=1.0), product of:
             0.083208114 = queryWeight, product of:
               1.6532879 = boost
               3.617579 = idf(docFreq=3226, maxDocs=44218)
               0.013912308 = queryNorm
             0.19783635 = fieldWeight in 4197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.617579 = idf(docFreq=3226, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.0592226 = weight(abstract_txt:measures in 4197) [ClassicSimilarity], result of:
           0.0592226 = score(doc=4197,freq=2.0), product of:
             0.14088127 = queryWeight, product of:
               1.863043 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.013912308 = queryNorm
             0.42037243 = fieldWeight in 4197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.02330176 = weight(abstract_txt:used in 4197) [ClassicSimilarity], result of:
           0.02330176 = score(doc=4197,freq=2.0), product of:
             0.08968855 = queryWeight, product of:
               1.9190626 = boost
               3.3592992 = idf(docFreq=4177, maxDocs=44218)
               0.013912308 = queryNorm
             0.25980753 = fieldWeight in 4197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3592992 = idf(docFreq=4177, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.053416975 = weight(abstract_txt:size in 4197) [ClassicSimilarity], result of:
           0.053416975 = score(doc=4197,freq=1.0), product of:
             0.16570073 = queryWeight, product of:
               2.0204983 = boost
               5.8947687 = idf(docFreq=330, maxDocs=44218)
               0.013912308 = queryNorm
             0.32237017 = fieldWeight in 4197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8947687 = idf(docFreq=330, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.15684482 = weight(abstract_txt:trec in 4197) [ClassicSimilarity], result of:
           0.15684482 = score(doc=4197,freq=1.0), product of:
             0.42808342 = queryWeight, product of:
               4.592778 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.013912308 = queryNorm
             0.36638847 = fieldWeight in 4197, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
         0.43784514 = weight(abstract_txt:pool in 4197) [ClassicSimilarity], result of:
           0.43784514 = score(doc=4197,freq=2.0), product of:
             0.70914364 = queryWeight, product of:
               6.3848658 = boost
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.013912308 = queryNorm
             0.617428 = fieldWeight in 4197, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4197)
       0.48 = coord(12/25)
   

2. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.34

   0.34153423 = sum of:
     0.34153423 = product of:
       0.8538355 = sum of:
         0.013658208 = weight(abstract_txt:these in 2591) [ClassicSimilarity], result of:
           0.013658208 = score(doc=2591,freq=2.0), product of:
             0.048468858 = queryWeight, product of:
               1.0927672 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.013912308 = queryNorm
             0.28179348 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.011837269 = weight(abstract_txt:systems in 2591) [ClassicSimilarity], result of:
           0.011837269 = score(doc=2591,freq=1.0), product of:
             0.055510864 = queryWeight, product of:
               1.1694597 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.013912308 = queryNorm
             0.2132424 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.033513553 = weight(abstract_txt:precision in 2591) [ClassicSimilarity], result of:
           0.033513553 = score(doc=2591,freq=1.0), product of:
             0.09704955 = queryWeight, product of:
               1.2625465 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.013912308 = queryNorm
             0.34532416 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.020824026 = weight(abstract_txt:each in 2591) [ClassicSimilarity], result of:
           0.020824026 = score(doc=2591,freq=1.0), product of:
             0.08089471 = queryWeight, product of:
               1.4117452 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.013912308 = queryNorm
             0.25742137 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.03613558 = weight(abstract_txt:documents in 2591) [ClassicSimilarity], result of:
           0.03613558 = score(doc=2591,freq=3.0), product of:
             0.080995314 = queryWeight, product of:
               1.4126228 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.013912308 = queryNorm
             0.44614407 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.02155294 = weight(abstract_txt:system in 2591) [ClassicSimilarity], result of:
           0.02155294 = score(doc=2591,freq=2.0), product of:
             0.072307646 = queryWeight, product of:
               1.5411955 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.013912308 = queryNorm
             0.2980728 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.023585295 = weight(abstract_txt:retrieval in 2591) [ClassicSimilarity], result of:
           0.023585295 = score(doc=2591,freq=2.0), product of:
             0.076784596 = queryWeight, product of:
               1.5881908 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013912308 = queryNorm
             0.3071618 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.085396804 = weight(abstract_txt:depth in 2591) [ClassicSimilarity], result of:
           0.085396804 = score(doc=2591,freq=1.0), product of:
             0.22811347 = queryWeight, product of:
               2.737418 = boost
               5.989777 = idf(docFreq=300, maxDocs=44218)
               0.013912308 = queryNorm
             0.37436107 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.989777 = idf(docFreq=300, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.25349954 = weight(abstract_txt:trec in 2591) [ClassicSimilarity], result of:
           0.25349954 = score(doc=2591,freq=2.0), product of:
             0.42808342 = queryWeight, product of:
               4.592778 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.013912308 = queryNorm
             0.5921732 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.3538323 = weight(abstract_txt:pool in 2591) [ClassicSimilarity], result of:
           0.3538323 = score(doc=2591,freq=1.0), product of:
             0.70914364 = queryWeight, product of:
               6.3848658 = boost
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.013912308 = queryNorm
             0.4989572 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
       0.4 = coord(10/25)
   

3. Losada, D.E.; Parapar, J.; Barreiro, A.: When to stop making relevance judgments? : a study of stopping methods for building information retrieval test collections (2019) 0.33

   0.33488613 = sum of:
     0.33488613 = product of:
       0.7611048 = sum of:
         0.013658208 = weight(abstract_txt:these in 4674) [ClassicSimilarity], result of:
           0.013658208 = score(doc=4674,freq=2.0), product of:
             0.048468858 = queryWeight, product of:
               1.0927672 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.013912308 = queryNorm
             0.28179348 = fieldWeight in 4674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.009808286 = weight(abstract_txt:have in 4674) [ClassicSimilarity], result of:
           0.009808286 = score(doc=4674,freq=1.0), product of:
             0.04897101 = queryWeight, product of:
               1.0984133 = boost
               3.2046018 = idf(docFreq=4876, maxDocs=44218)
               0.013912308 = queryNorm
             0.20028761 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.2046018 = idf(docFreq=4876, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.011837269 = weight(abstract_txt:systems in 4674) [ClassicSimilarity], result of:
           0.011837269 = score(doc=4674,freq=1.0), product of:
             0.055510864 = queryWeight, product of:
               1.1694597 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.013912308 = queryNorm
             0.2132424 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.020758033 = weight(abstract_txt:different in 4674) [ClassicSimilarity], result of:
           0.020758033 = score(doc=4674,freq=2.0), product of:
             0.06407045 = queryWeight, product of:
               1.256392 = boost
               3.6655018 = idf(docFreq=3075, maxDocs=44218)
               0.013912308 = queryNorm
             0.32398763 = fieldWeight in 4674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6655018 = idf(docFreq=3075, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.047395322 = weight(abstract_txt:precision in 4674) [ClassicSimilarity], result of:
           0.047395322 = score(doc=4674,freq=2.0), product of:
             0.09704955 = queryWeight, product of:
               1.2625465 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.013912308 = queryNorm
             0.4883621 = fieldWeight in 4674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.037750535 = weight(abstract_txt:recall in 4674) [ClassicSimilarity], result of:
           0.037750535 = score(doc=4674,freq=1.0), product of:
             0.105065934 = queryWeight, product of:
               1.3136559 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.013912308 = queryNorm
             0.35930327 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.03613558 = weight(abstract_txt:documents in 4674) [ClassicSimilarity], result of:
           0.03613558 = score(doc=4674,freq=3.0), product of:
             0.080995314 = queryWeight, product of:
               1.4126228 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.013912308 = queryNorm
             0.44614407 = fieldWeight in 4674, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.016677324 = weight(abstract_txt:retrieval in 4674) [ClassicSimilarity], result of:
           0.016677324 = score(doc=4674,freq=1.0), product of:
             0.076784596 = queryWeight, product of:
               1.5881908 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013912308 = queryNorm
             0.21719621 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.04785909 = weight(abstract_txt:measures in 4674) [ClassicSimilarity], result of:
           0.04785909 = score(doc=4674,freq=1.0), product of:
             0.14088127 = queryWeight, product of:
               1.863043 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.013912308 = queryNorm
             0.33971223 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.018830666 = weight(abstract_txt:used in 4674) [ClassicSimilarity], result of:
           0.018830666 = score(doc=4674,freq=1.0), product of:
             0.08968855 = queryWeight, product of:
               1.9190626 = boost
               3.3592992 = idf(docFreq=4177, maxDocs=44218)
               0.013912308 = queryNorm
             0.2099562 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3592992 = idf(docFreq=4177, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.50039446 = weight(abstract_txt:pool in 4674) [ClassicSimilarity], result of:
           0.50039446 = score(doc=4674,freq=2.0), product of:
             0.70914364 = queryWeight, product of:
               6.3848658 = boost
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.013912308 = queryNorm
             0.70563203 = fieldWeight in 4674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.983315 = idf(docFreq=40, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
       0.44 = coord(11/25)
   

4. Voorhees, E.M.; Harman, D.K.: ¬The Text REtrieval Conference (2005) 0.32

   0.32057205 = sum of:
     0.32057205 = product of:
       0.6678584 = sum of:
         0.0060361326 = weight(abstract_txt:these in 5082) [ClassicSimilarity], result of:
           0.0060361326 = score(doc=5082,freq=1.0), product of:
             0.048468858 = queryWeight, product of:
               1.0927672 = boost
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.013912308 = queryNorm
             0.124536306 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1881294 = idf(docFreq=4957, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.01621893 = weight(abstract_txt:have in 5082) [ClassicSimilarity], result of:
           0.01621893 = score(doc=5082,freq=7.0), product of:
             0.04897101 = queryWeight, product of:
               1.0984133 = boost
               3.2046018 = idf(docFreq=4876, maxDocs=44218)
               0.013912308 = queryNorm
             0.33119452 = fieldWeight in 5082, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.2046018 = idf(docFreq=4876, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.014404008 = weight(abstract_txt:individual in 5082) [ClassicSimilarity], result of:
           0.014404008 = score(doc=5082,freq=1.0), product of:
             0.075610496 = queryWeight, product of:
               1.1144015 = boost
               4.8768706 = idf(docFreq=915, maxDocs=44218)
               0.013912308 = queryNorm
             0.19050276 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8768706 = idf(docFreq=915, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.010462767 = weight(abstract_txt:systems in 5082) [ClassicSimilarity], result of:
           0.010462767 = score(doc=5082,freq=2.0), product of:
             0.055510864 = queryWeight, product of:
               1.1694597 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.013912308 = queryNorm
             0.18848144 = fieldWeight in 5082, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.009173841 = weight(abstract_txt:different in 5082) [ClassicSimilarity], result of:
           0.009173841 = score(doc=5082,freq=1.0), product of:
             0.06407045 = queryWeight, product of:
               1.256392 = boost
               3.6655018 = idf(docFreq=3075, maxDocs=44218)
               0.013912308 = queryNorm
             0.14318366 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6655018 = idf(docFreq=3075, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.013015017 = weight(abstract_txt:each in 5082) [ClassicSimilarity], result of:
           0.013015017 = score(doc=5082,freq=1.0), product of:
             0.08089471 = queryWeight, product of:
               1.4117452 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.013912308 = queryNorm
             0.16088836 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.013039305 = weight(abstract_txt:documents in 5082) [ClassicSimilarity], result of:
           0.013039305 = score(doc=5082,freq=1.0), product of:
             0.080995314 = queryWeight, product of:
               1.4126228 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.013912308 = queryNorm
             0.16098839 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.037231505 = weight(abstract_txt:evaluations in 5082) [ClassicSimilarity], result of:
           0.037231505 = score(doc=5082,freq=1.0), product of:
             0.14240775 = queryWeight, product of:
               1.5293871 = boost
               6.6929407 = idf(docFreq=148, maxDocs=44218)
               0.013912308 = queryNorm
             0.261443 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6929407 = idf(docFreq=148, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.039000515 = weight(abstract_txt:retrieval in 5082) [ClassicSimilarity], result of:
           0.039000515 = score(doc=5082,freq=14.0), product of:
             0.076784596 = queryWeight, product of:
               1.5881908 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013912308 = queryNorm
             0.5079211 = fieldWeight in 5082, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.01855477 = weight(abstract_txt:relevant in 5082) [ClassicSimilarity], result of:
           0.01855477 = score(doc=5082,freq=1.0), product of:
             0.10246935 = queryWeight, product of:
               1.5888878 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.013912308 = queryNorm
             0.18107629 = fieldWeight in 5082, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.028801782 = weight(abstract_txt:been in 5082) [ClassicSimilarity], result of:
           0.028801782 = score(doc=5082,freq=6.0), product of:
             0.083208114 = queryWeight, product of:
               1.6532879 = boost
               3.617579 = idf(docFreq=3226, maxDocs=44218)
               0.013912308 = queryNorm
             0.34614152 = fieldWeight in 5082, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.617579 = idf(docFreq=3226, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
         0.46191987 = weight(abstract_txt:trec in 5082) [ClassicSimilarity], result of:
           0.46191987 = score(doc=5082,freq=17.0), product of:
             0.42808342 = queryWeight, product of:
               4.592778 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.013912308 = queryNorm
             1.0790417 = fieldWeight in 5082, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0390625 = fieldNorm(doc=5082)
       0.48 = coord(12/25)
   

5. Kantor, P.; Kim, M.H.; Ibraev, U.; Atasoy, K.: Estimating the number of relevant documents in enormous collections (1999) 0.28

   0.283445 = sum of:
     0.283445 = product of:
       0.6441932 = sum of:
         0.01972608 = weight(abstract_txt:performance in 6690) [ClassicSimilarity], result of:
           0.01972608 = score(doc=6690,freq=1.0), product of:
             0.068161696 = queryWeight, product of:
               1.0580857 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.013912308 = queryNorm
             0.28940126 = fieldWeight in 6690, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.023046413 = weight(abstract_txt:individual in 6690) [ClassicSimilarity], result of:
           0.023046413 = score(doc=6690,freq=1.0), product of:
             0.075610496 = queryWeight, product of:
               1.1144015 = boost
               4.8768706 = idf(docFreq=915, maxDocs=44218)
               0.013912308 = queryNorm
             0.3048044 = fieldWeight in 6690, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8768706 = idf(docFreq=915, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.020502752 = weight(abstract_txt:systems in 6690) [ClassicSimilarity], result of:
           0.020502752 = score(doc=6690,freq=3.0), product of:
             0.055510864 = queryWeight, product of:
               1.1694597 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.013912308 = queryNorm
             0.36934665 = fieldWeight in 6690, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.033513553 = weight(abstract_txt:precision in 6690) [ClassicSimilarity], result of:
           0.033513553 = score(doc=6690,freq=1.0), product of:
             0.09704955 = queryWeight, product of:
               1.2625465 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.013912308 = queryNorm
             0.34532416 = fieldWeight in 6690, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.02944962 = weight(abstract_txt:each in 6690) [ClassicSimilarity], result of:
           0.02944962 = score(doc=6690,freq=2.0), product of:
             0.08089471 = queryWeight, product of:
               1.4117452 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.013912308 = queryNorm
             0.36404878 = fieldWeight in 6690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.046650838 = weight(abstract_txt:documents in 6690) [ClassicSimilarity], result of:
           0.046650838 = score(doc=6690,freq=5.0), product of:
             0.080995314 = queryWeight, product of:
               1.4126228 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.013912308 = queryNorm
             0.5759696 = fieldWeight in 6690, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.02155294 = weight(abstract_txt:system in 6690) [ClassicSimilarity], result of:
           0.02155294 = score(doc=6690,freq=2.0), product of:
             0.072307646 = queryWeight, product of:
               1.5411955 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.013912308 = queryNorm
             0.2980728 = fieldWeight in 6690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.023585295 = weight(abstract_txt:retrieval in 6690) [ClassicSimilarity], result of:
           0.023585295 = score(doc=6690,freq=2.0), product of:
             0.076784596 = queryWeight, product of:
               1.5881908 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013912308 = queryNorm
             0.3071618 = fieldWeight in 6690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.089062884 = weight(abstract_txt:relevant in 6690) [ClassicSimilarity], result of:
           0.089062884 = score(doc=6690,freq=9.0), product of:
             0.10246935 = queryWeight, product of:
               1.5888878 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.013912308 = queryNorm
             0.86916614 = fieldWeight in 6690, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.026630582 = weight(abstract_txt:used in 6690) [ClassicSimilarity], result of:
           0.026630582 = score(doc=6690,freq=2.0), product of:
             0.08968855 = queryWeight, product of:
               1.9190626 = boost
               3.3592992 = idf(docFreq=4177, maxDocs=44218)
               0.013912308 = queryNorm
             0.2969229 = fieldWeight in 6690, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.3592992 = idf(docFreq=4177, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
         0.31047225 = weight(abstract_txt:trec in 6690) [ClassicSimilarity], result of:
           0.31047225 = score(doc=6690,freq=3.0), product of:
             0.42808342 = queryWeight, product of:
               4.592778 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.013912308 = queryNorm
             0.7252611 = fieldWeight in 6690, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=6690)
       0.44 = coord(11/25)