Document (#39674)

Author
Kenter, T.
Balog, K.
Rijke, M. de
Title
Evaluating document filtering systems over time
Source
Information processing and management. 51(2015) no.6, S.791-808
Year
2015
Abstract
Document filtering is a popular task in information retrieval. A stream of documents arriving over time is filtered for documents relevant to a set of topics. The distinguishing feature of document filtering is the temporal aspect introduced by the stream of documents. Document filtering systems, up to now, have been evaluated in terms of traditional metrics like (micro- or macro-averaged) precision, recall, MAP, nDCG, F1 and utility. We argue that these metrics do not capture all relevant aspects of the systems being evaluated. In particular, they lack support for the temporal dimension of the task. We propose a time-sensitive way of measuring performance of document filtering systems over time by employing trend estimation. In short, the performance is calculated for batches, a trend line is fitted to the results, and the estimated performance of systems at the end of the evaluation period is used to compare systems. We detail the application of our proposed trend estimation framework and examine the assumptions that need to hold for valid significance testing. Additionally, we analyze the requirements a document filtering metric has to meet and show that traditional macro-averaged true-positive-based metrics, like precision, recall and utility fail to capture essential information when applied in a batch setting. In particular, false positives returned in a batch for topics that are absent from the ground truth in that batch go unnoticed. This is a serious flaw as over-generation of a system might be overlooked this way. We propose a new metric, aptness, that does capture false positives. We incorporate this metric in an overall score and show that this new score does meet all requirements. To demonstrate the results of our proposed evaluation methodology, we analyze the runs submitted to the two most recent editions of a document filtering evaluation campaign. We re-evaluate the runs submitted to the Cumulative Citation Recommendation task of the 2012 and 2013 editions of the TREC Knowledge Base Acceleration track, and show that important new insights emerge.
Content
Vgl.: doi:10.1016/j.ipm.2015.03.005.
Footnote
Beitrag in einem Themenschwerpunkt "Time and information retrieval"

Similar documents (author)

  1. Balog, K.; Azzopardi, L.; Rijke, M. de: ¬A language modeling framework for expert finding (2009) 4.92
    4.922386 = sum of:
      4.922386 = sum of:
        1.9875885 = weight(author_txt:rijke in 4448) [ClassicSimilarity], result of:
          1.9875885 = score(doc=4448,freq=1.0), product of:
            0.6106887 = queryWeight, product of:
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.07036304 = queryNorm
            3.2546673 = fieldWeight in 4448, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.375 = fieldNorm(doc=4448)
        2.9347975 = weight(author_txt:balog in 4448) [ClassicSimilarity], result of:
          2.9347975 = score(doc=4448,freq=1.0), product of:
            0.79187083 = queryWeight, product of:
              1.1387208 = boost
              9.883085 = idf(docFreq=5, maxDocs=43254)
              0.07036304 = queryNorm
            3.706157 = fieldWeight in 4448, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.883085 = idf(docFreq=5, maxDocs=43254)
              0.375 = fieldNorm(doc=4448)
    
  2. Hofmann, K.; Balog, K.; Bogers, T.; Rijke, M. de: Contextual factors for finding similar experts (2010) 4.10
    4.101989 = sum of:
      4.101989 = sum of:
        1.6563237 = weight(author_txt:rijke in 457) [ClassicSimilarity], result of:
          1.6563237 = score(doc=457,freq=1.0), product of:
            0.6106887 = queryWeight, product of:
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.07036304 = queryNorm
            2.7122226 = fieldWeight in 457, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.3125 = fieldNorm(doc=457)
        2.445665 = weight(author_txt:balog in 457) [ClassicSimilarity], result of:
          2.445665 = score(doc=457,freq=1.0), product of:
            0.79187083 = queryWeight, product of:
              1.1387208 = boost
              9.883085 = idf(docFreq=5, maxDocs=43254)
              0.07036304 = queryNorm
            3.0884643 = fieldWeight in 457, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.883085 = idf(docFreq=5, maxDocs=43254)
              0.3125 = fieldNorm(doc=457)
    
  3. Berendsen, R.; Rijke, M. de; Balog, K.; Bogers, T.; Bosch, A. van den: On the assessment of expertise profiles (2013) 3.28
    3.2815907 = sum of:
      3.2815907 = sum of:
        1.3250589 = weight(author_txt:rijke in 2554) [ClassicSimilarity], result of:
          1.3250589 = score(doc=2554,freq=1.0), product of:
            0.6106887 = queryWeight, product of:
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.07036304 = queryNorm
            2.169778 = fieldWeight in 2554, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.25 = fieldNorm(doc=2554)
        1.9565318 = weight(author_txt:balog in 2554) [ClassicSimilarity], result of:
          1.9565318 = score(doc=2554,freq=1.0), product of:
            0.79187083 = queryWeight, product of:
              1.1387208 = boost
              9.883085 = idf(docFreq=5, maxDocs=43254)
              0.07036304 = queryNorm
            2.4707713 = fieldWeight in 2554, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.883085 = idf(docFreq=5, maxDocs=43254)
              0.25 = fieldNorm(doc=2554)
    
  4. De Rijke, M. -> Rijke, M. de: 1.41
    1.4054374 = sum of:
      1.4054374 = product of:
        2.8108747 = sum of:
          2.8108747 = weight(author_txt:rijke in 117) [ClassicSimilarity], result of:
            2.8108747 = score(doc=117,freq=2.0), product of:
              0.6106887 = queryWeight, product of:
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.07036304 = queryNorm
              4.6027946 = fieldWeight in 117, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.375 = fieldNorm(doc=117)
        0.5 = coord(1/2)
    
  5. Li, X.; Rijke, M.de: Characterizing and predicting downloads in academic search (2019) 1.33
    1.3250589 = sum of:
      1.3250589 = product of:
        2.6501179 = sum of:
          2.6501179 = weight(author_txt:rijke in 104) [ClassicSimilarity], result of:
            2.6501179 = score(doc=104,freq=1.0), product of:
              0.6106887 = queryWeight, product of:
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.07036304 = queryNorm
              4.339556 = fieldWeight in 104, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.5 = fieldNorm(doc=104)
        0.5 = coord(1/2)
    

Similar documents (content)

  1. Sakai, T.: On the reliability of information retrieval metrics based on graded relevance (2007) 0.16
    0.15905961 = sum of:
      0.15905961 = product of:
        0.6627484 = sum of:
          0.08139466 = weight(abstract_txt:submitted in 2911) [ClassicSimilarity], result of:
            0.08139466 = score(doc=2911,freq=1.0), product of:
              0.1487931 = queryWeight, product of:
                1.0395961 = boost
                7.002016 = idf(docFreq=106, maxDocs=43254)
                0.020440668 = queryNorm
              0.5470325 = fieldWeight in 2911, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.002016 = idf(docFreq=106, maxDocs=43254)
                0.078125 = fieldNorm(doc=2911)
          0.11452635 = weight(abstract_txt:runs in 2911) [ClassicSimilarity], result of:
            0.11452635 = score(doc=2911,freq=1.0), product of:
              0.18683395 = queryWeight, product of:
                1.1649334 = boost
                7.846204 = idf(docFreq=45, maxDocs=43254)
                0.020440668 = queryNorm
              0.61298466 = fieldWeight in 2911, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.846204 = idf(docFreq=45, maxDocs=43254)
                0.078125 = fieldNorm(doc=2911)
          0.022296658 = weight(abstract_txt:that in 2911) [ClassicSimilarity], result of:
            0.022296658 = score(doc=2911,freq=3.0), product of:
              0.069075644 = queryWeight, product of:
                1.4166602 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.020440668 = queryNorm
              0.3227861 = fieldWeight in 2911, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.078125 = fieldNorm(doc=2911)
          0.23607826 = weight(abstract_txt:metrics in 2911) [ClassicSimilarity], result of:
            0.23607826 = score(doc=2911,freq=5.0), product of:
              0.20258002 = queryWeight, product of:
                1.4856522 = boost
                6.670899 = idf(docFreq=148, maxDocs=43254)
                0.020440668 = queryNorm
              1.1653581 = fieldWeight in 2911, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                6.670899 = idf(docFreq=148, maxDocs=43254)
                0.078125 = fieldNorm(doc=2911)
          0.14320593 = weight(abstract_txt:metric in 2911) [ClassicSimilarity], result of:
            0.14320593 = score(doc=2911,freq=1.0), product of:
              0.24823132 = queryWeight, product of:
                1.6445502 = boost
                7.3843856 = idf(docFreq=72, maxDocs=43254)
                0.020440668 = queryNorm
              0.57690513 = fieldWeight in 2911, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.3843856 = idf(docFreq=72, maxDocs=43254)
                0.078125 = fieldNorm(doc=2911)
          0.0652465 = weight(abstract_txt:document in 2911) [ClassicSimilarity], result of:
            0.0652465 = score(doc=2911,freq=1.0), product of:
              0.19494541 = queryWeight, product of:
                2.2261977 = boost
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.020440668 = queryNorm
              0.33469114 = fieldWeight in 2911, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.078125 = fieldNorm(doc=2911)
        0.24 = coord(6/25)
    
  2. Lai, C,-H.: Applying knowledge flow mining to group recommendation methods for task-based groups (2015) 0.15
    0.15499233 = sum of:
      0.15499233 = product of:
        0.55354404 = sum of:
          0.040138748 = weight(abstract_txt:performance in 3132) [ClassicSimilarity], result of:
            0.040138748 = score(doc=3132,freq=2.0), product of:
              0.097916335 = queryWeight, product of:
                1.0328718 = boost
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.020440668 = queryNorm
              0.409929 = fieldWeight in 3132, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
          0.083249085 = weight(abstract_txt:task in 3132) [ClassicSimilarity], result of:
            0.083249085 = score(doc=3132,freq=6.0), product of:
              0.11041426 = queryWeight, product of:
                1.0968101 = boost
                4.924914 = idf(docFreq=853, maxDocs=43254)
                0.020440668 = queryNorm
              0.7539704 = fieldWeight in 3132, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                4.924914 = idf(docFreq=853, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
          0.038593538 = weight(abstract_txt:time in 3132) [ClassicSimilarity], result of:
            0.038593538 = score(doc=3132,freq=2.0), product of:
              0.104986936 = queryWeight, product of:
                1.2349684 = boost
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.020440668 = queryNorm
              0.36760324 = fieldWeight in 3132, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
          0.04137803 = weight(abstract_txt:over in 3132) [ClassicSimilarity], result of:
            0.04137803 = score(doc=3132,freq=2.0), product of:
              0.10997789 = queryWeight, product of:
                1.263982 = boost
                4.2566643 = idf(docFreq=1665, maxDocs=43254)
                0.020440668 = queryNorm
              0.3762395 = fieldWeight in 3132, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.2566643 = idf(docFreq=1665, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
          0.014564116 = weight(abstract_txt:that in 3132) [ClassicSimilarity], result of:
            0.014564116 = score(doc=3132,freq=2.0), product of:
              0.069075644 = queryWeight, product of:
                1.4166602 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.020440668 = queryNorm
              0.210843 = fieldWeight in 3132, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
          0.07381798 = weight(abstract_txt:document in 3132) [ClassicSimilarity], result of:
            0.07381798 = score(doc=3132,freq=2.0), product of:
              0.19494541 = queryWeight, product of:
                2.2261977 = boost
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.020440668 = queryNorm
              0.37865978 = fieldWeight in 3132, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
          0.26180255 = weight(abstract_txt:filtering in 3132) [ClassicSimilarity], result of:
            0.26180255 = score(doc=3132,freq=2.0), product of:
              0.4533714 = queryWeight, product of:
                3.3949573 = boost
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.020440668 = queryNorm
              0.57745713 = fieldWeight in 3132, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.0625 = fieldNorm(doc=3132)
        0.28 = coord(7/25)
    
  3. Ku, C.-H.; Leroy, G.: ¬A crime reports analysis system to identify related crimes (2011) 0.14
    0.14148207 = sum of:
      0.14148207 = product of:
        0.44213146 = sum of:
          0.025757845 = weight(abstract_txt:evaluation in 1094) [ClassicSimilarity], result of:
            0.025757845 = score(doc=1094,freq=1.0), product of:
              0.09178301 = queryWeight, product of:
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.020440668 = queryNorm
              0.2806385 = fieldWeight in 1094, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.02838238 = weight(abstract_txt:performance in 1094) [ClassicSimilarity], result of:
            0.02838238 = score(doc=1094,freq=1.0), product of:
              0.097916335 = queryWeight, product of:
                1.0328718 = boost
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.020440668 = queryNorm
              0.2898636 = fieldWeight in 1094, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6378174 = idf(docFreq=1137, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.033986297 = weight(abstract_txt:task in 1094) [ClassicSimilarity], result of:
            0.033986297 = score(doc=1094,freq=1.0), product of:
              0.11041426 = queryWeight, product of:
                1.0968101 = boost
                4.924914 = idf(docFreq=853, maxDocs=43254)
                0.020440668 = queryNorm
              0.30780712 = fieldWeight in 1094, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.924914 = idf(docFreq=853, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.027289754 = weight(abstract_txt:time in 1094) [ClassicSimilarity], result of:
            0.027289754 = score(doc=1094,freq=1.0), product of:
              0.104986936 = queryWeight, product of:
                1.2349684 = boost
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.020440668 = queryNorm
              0.25993475 = fieldWeight in 1094, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.014564116 = weight(abstract_txt:that in 1094) [ClassicSimilarity], result of:
            0.014564116 = score(doc=1094,freq=2.0), product of:
              0.069075644 = queryWeight, product of:
                1.4166602 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.020440668 = queryNorm
              0.210843 = fieldWeight in 1094, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.022634292 = weight(abstract_txt:systems in 1094) [ClassicSimilarity], result of:
            0.022634292 = score(doc=1094,freq=1.0), product of:
              0.10609067 = queryWeight, product of:
                1.5204511 = boost
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.020440668 = queryNorm
              0.21334855 = fieldWeight in 1094, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.104394406 = weight(abstract_txt:document in 1094) [ClassicSimilarity], result of:
            0.104394406 = score(doc=1094,freq=4.0), product of:
              0.19494541 = queryWeight, product of:
                2.2261977 = boost
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.020440668 = queryNorm
              0.53550583 = fieldWeight in 1094, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
          0.18512236 = weight(abstract_txt:filtering in 1094) [ClassicSimilarity], result of:
            0.18512236 = score(doc=1094,freq=1.0), product of:
              0.4533714 = queryWeight, product of:
                3.3949573 = boost
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.020440668 = queryNorm
              0.40832385 = fieldWeight in 1094, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.0625 = fieldNorm(doc=1094)
        0.32 = coord(8/25)
    
  4. Pera, M.S.; Ng, Y.-K.: SpamED : a spam E-mail detection approach based on phrase similarity (2009) 0.13
    0.13020633 = sum of:
      0.13020633 = product of:
        0.6510316 = sum of:
          0.14904639 = weight(abstract_txt:false in 4722) [ClassicSimilarity], result of:
            0.14904639 = score(doc=4722,freq=2.0), product of:
              0.17676237 = queryWeight, product of:
                1.1330997 = boost
                7.6317935 = idf(docFreq=56, maxDocs=43254)
                0.020440668 = queryNorm
              0.843202 = fieldWeight in 4722, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.6317935 = idf(docFreq=56, maxDocs=43254)
                0.078125 = fieldNorm(doc=4722)
          0.048241924 = weight(abstract_txt:time in 4722) [ClassicSimilarity], result of:
            0.048241924 = score(doc=4722,freq=2.0), product of:
              0.104986936 = queryWeight, product of:
                1.2349684 = boost
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.020440668 = queryNorm
              0.45950407 = fieldWeight in 4722, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.078125 = fieldNorm(doc=4722)
          0.012872982 = weight(abstract_txt:that in 4722) [ClassicSimilarity], result of:
            0.012872982 = score(doc=4722,freq=1.0), product of:
              0.069075644 = queryWeight, product of:
                1.4166602 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.020440668 = queryNorm
              0.18636064 = fieldWeight in 4722, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.078125 = fieldNorm(doc=4722)
          0.20946737 = weight(abstract_txt:positives in 4722) [ClassicSimilarity], result of:
            0.20946737 = score(doc=4722,freq=1.0), product of:
              0.2794236 = queryWeight, product of:
                1.4246387 = boost
                9.595404 = idf(docFreq=7, maxDocs=43254)
                0.020440668 = queryNorm
              0.74964094 = fieldWeight in 4722, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.595404 = idf(docFreq=7, maxDocs=43254)
                0.078125 = fieldNorm(doc=4722)
          0.23140295 = weight(abstract_txt:filtering in 4722) [ClassicSimilarity], result of:
            0.23140295 = score(doc=4722,freq=1.0), product of:
              0.4533714 = queryWeight, product of:
                3.3949573 = boost
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.020440668 = queryNorm
              0.5104048 = fieldWeight in 4722, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.078125 = fieldNorm(doc=4722)
        0.2 = coord(5/25)
    
  5. Toepfer, M.; Seifert, C.: Content-based quality estimation for automatic subject indexing of short texts under precision and recall constraints 0.13
    0.12596665 = sum of:
      0.12596665 = product of:
        0.52486104 = sum of:
          0.116483904 = weight(abstract_txt:estimation in 310) [ClassicSimilarity], result of:
            0.116483904 = score(doc=310,freq=1.0), product of:
              0.18895692 = queryWeight, product of:
                1.1715332 = boost
                7.8906555 = idf(docFreq=43, maxDocs=43254)
                0.020440668 = queryNorm
              0.61645746 = fieldWeight in 310, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.8906555 = idf(docFreq=43, maxDocs=43254)
                0.078125 = fieldNorm(doc=310)
          0.034112193 = weight(abstract_txt:time in 310) [ClassicSimilarity], result of:
            0.034112193 = score(doc=310,freq=1.0), product of:
              0.104986936 = queryWeight, product of:
                1.2349684 = boost
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.020440668 = queryNorm
              0.32491845 = fieldWeight in 310, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.158956 = idf(docFreq=1836, maxDocs=43254)
                0.078125 = fieldNorm(doc=310)
          0.022296658 = weight(abstract_txt:that in 310) [ClassicSimilarity], result of:
            0.022296658 = score(doc=310,freq=3.0), product of:
              0.069075644 = queryWeight, product of:
                1.4166602 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.020440668 = queryNorm
              0.3227861 = fieldWeight in 310, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.078125 = fieldNorm(doc=310)
          0.028292865 = weight(abstract_txt:systems in 310) [ClassicSimilarity], result of:
            0.028292865 = score(doc=310,freq=1.0), product of:
              0.10609067 = queryWeight, product of:
                1.5204511 = boost
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.020440668 = queryNorm
              0.2666857 = fieldWeight in 310, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.078125 = fieldNorm(doc=310)
          0.09227248 = weight(abstract_txt:document in 310) [ClassicSimilarity], result of:
            0.09227248 = score(doc=310,freq=2.0), product of:
              0.19494541 = queryWeight, product of:
                2.2261977 = boost
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.020440668 = queryNorm
              0.47332472 = fieldWeight in 310, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.078125 = fieldNorm(doc=310)
          0.23140295 = weight(abstract_txt:filtering in 310) [ClassicSimilarity], result of:
            0.23140295 = score(doc=310,freq=1.0), product of:
              0.4533714 = queryWeight, product of:
                3.3949573 = boost
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.020440668 = queryNorm
              0.5104048 = fieldWeight in 310, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.5331817 = idf(docFreq=170, maxDocs=43254)
                0.078125 = fieldNorm(doc=310)
        0.24 = coord(6/25)