Document (#42050)

Author

Angelini, M.
Fazzini, V.
Ferro, N.
Santucci, G.
Silvello, G.

Title

CLAIRE: A combinatorial visual analytics system for information retrieval evaluation

Source

Information processing and management. 54(2018) no.6, S.1077-1100

Year

2018

Abstract

Information Retrieval (IR) develops complex systems, constituted of several components, which aim at returning and optimally ranking the most relevant documents in response to user queries. In this context, experimental evaluation plays a central role, since it allows for measuring IR systems effectiveness, increasing the understanding of their functioning, and better directing the efforts for improving them. Current evaluation methodologies are limited by two major factors: (i) IR systems are evaluated as "black boxes", since it is not possible to decompose the contributions of the different components, e.g., stop lists, stemmers, and IR models; (ii) given that it is not possible to predict the effectiveness of an IR system, both academia and industry need to explore huge numbers of systems, originated by large combinatorial compositions of their components, to understand how they perform and how these components interact together. We propose a Combinatorial visuaL Analytics system for Information Retrieval Evaluation (CLAIRE) which allows for exploring and making sense of the performances of a large amount of IR systems, in order to quickly and intuitively grasp which system configurations are preferred, what are the contributions of the different components and how these components interact together. The CLAIRE system is then validated against use cases based on several test collections using a wide set of systems, generated by a combinatorial composition of several off-the-shelf components, representing the most common denominator almost always present in English IR systems. In particular, we validate the findings enabled by CLAIRE with respect to consolidated deep statistical analyses and we show that the CLAIRE system allows the generation of new insights, which were not detectable with traditional approaches.

Content

Vgl.: https://doi.org/10.1016/j.ipm.2018.04.006.

Theme

Retrievalstudien

Similar documents (author)

1. Ferro, N.; Silvello, G.: NESTOR: a formal model for digital archives (2013) 6.85

   6.8477716 = sum of:
     6.8477716 = sum of:
       3.353079 = weight(author_txt:silvello in 2707) [ClassicSimilarity], result of:
         3.353079 = score(doc=2707,freq=1.0), product of:
           0.69729096 = queryWeight, product of:
             9.617446 = idf(docFreq=7, maxDocs=44218)
             0.07250272 = queryNorm
           4.808723 = fieldWeight in 2707, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.617446 = idf(docFreq=7, maxDocs=44218)
             0.5 = fieldNorm(doc=2707)
       3.4946926 = weight(author_txt:ferro in 2707) [ClassicSimilarity], result of:
         3.4946926 = score(doc=2707,freq=1.0), product of:
           0.7167882 = queryWeight, product of:
             1.0138843 = boost
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.07250272 = queryNorm
           4.8754888 = fieldWeight in 2707, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.5 = fieldNorm(doc=2707)
   

2. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 6.85

   6.8477716 = sum of:
     6.8477716 = sum of:
       3.353079 = weight(author_txt:silvello in 4035) [ClassicSimilarity], result of:
         3.353079 = score(doc=4035,freq=1.0), product of:
           0.69729096 = queryWeight, product of:
             9.617446 = idf(docFreq=7, maxDocs=44218)
             0.07250272 = queryNorm
           4.808723 = fieldWeight in 4035, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.617446 = idf(docFreq=7, maxDocs=44218)
             0.5 = fieldNorm(doc=4035)
       3.4946926 = weight(author_txt:ferro in 4035) [ClassicSimilarity], result of:
         3.4946926 = score(doc=4035,freq=1.0), product of:
           0.7167882 = queryWeight, product of:
             1.0138843 = boost
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.07250272 = queryNorm
           4.8754888 = fieldWeight in 4035, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.5 = fieldNorm(doc=4035)
   

3. Ferro, N.; Silvello, G.; Keskustalo, H.; Pirkola, A.; Järvelin, K.: ¬The twist measure for IR evaluation : taking user's effort into account (2016) 4.28

   4.2798576 = sum of:
     4.2798576 = sum of:
       2.0956745 = weight(author_txt:silvello in 2771) [ClassicSimilarity], result of:
         2.0956745 = score(doc=2771,freq=1.0), product of:
           0.69729096 = queryWeight, product of:
             9.617446 = idf(docFreq=7, maxDocs=44218)
             0.07250272 = queryNorm
           3.005452 = fieldWeight in 2771, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.617446 = idf(docFreq=7, maxDocs=44218)
             0.3125 = fieldNorm(doc=2771)
       2.184183 = weight(author_txt:ferro in 2771) [ClassicSimilarity], result of:
         2.184183 = score(doc=2771,freq=1.0), product of:
           0.7167882 = queryWeight, product of:
             1.0138843 = boost
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.07250272 = queryNorm
           3.0471804 = fieldWeight in 2771, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.7509775 = idf(docFreq=6, maxDocs=44218)
             0.3125 = fieldNorm(doc=2771)
   

4. Silvello, G.: Learning to cite framework : how to automatically construct citations for hierarchical data (2017) 2.10

   2.0956745 = sum of:
     2.0956745 = product of:
       4.191349 = sum of:
         4.191349 = weight(author_txt:silvello in 3647) [ClassicSimilarity], result of:
           4.191349 = score(doc=3647,freq=1.0), product of:
             0.69729096 = queryWeight, product of:
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.07250272 = queryNorm
             6.010904 = fieldWeight in 3647, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.625 = fieldNorm(doc=3647)
       0.5 = coord(1/2)
   

5. Silvello, G.: Theory and practice of data citation (2018) 2.10

   2.0956745 = sum of:
     2.0956745 = product of:
       4.191349 = sum of:
         4.191349 = weight(author_txt:silvello in 4006) [ClassicSimilarity], result of:
           4.191349 = score(doc=4006,freq=1.0), product of:
             0.69729096 = queryWeight, product of:
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.07250272 = queryNorm
             6.010904 = fieldWeight in 4006, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.625 = fieldNorm(doc=4006)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 0.27

   0.2679307 = sum of:
     0.2679307 = product of:
       0.7442519 = sum of:
         0.08741445 = weight(abstract_txt:stemmers in 4035) [ClassicSimilarity], result of:
           0.08741445 = score(doc=4035,freq=1.0), product of:
             0.15441129 = queryWeight, product of:
               1.0149627 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.01679596 = queryNorm
             0.56611437 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.020944193 = weight(abstract_txt:retrieval in 4035) [ClassicSimilarity], result of:
           0.020944193 = score(doc=4035,freq=2.0), product of:
             0.068186186 = queryWeight, product of:
               1.1682059 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.01679596 = queryNorm
             0.3071618 = fieldWeight in 4035, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.05910954 = weight(abstract_txt:together in 4035) [ClassicSimilarity], result of:
           0.05910954 = score(doc=4035,freq=3.0), product of:
             0.103919774 = queryWeight, product of:
               1.1775372 = boost
               5.254347 = idf(docFreq=627, maxDocs=44218)
               0.01679596 = queryNorm
             0.56879973 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.254347 = idf(docFreq=627, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.011674884 = weight(abstract_txt:which in 4035) [ClassicSimilarity], result of:
           0.011674884 = score(doc=4035,freq=1.0), product of:
             0.06404402 = queryWeight, product of:
               1.3073138 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.01679596 = queryNorm
             0.18229467 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.066806145 = weight(abstract_txt:interact in 4035) [ClassicSimilarity], result of:
           0.066806145 = score(doc=4035,freq=1.0), product of:
             0.16262145 = queryWeight, product of:
               1.4730397 = boost
               6.572923 = idf(docFreq=167, maxDocs=44218)
               0.01679596 = queryNorm
             0.4108077 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.572923 = idf(docFreq=167, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.07357481 = weight(abstract_txt:evaluation in 4035) [ClassicSimilarity], result of:
           0.07357481 = score(doc=4035,freq=3.0), product of:
             0.15150347 = queryWeight, product of:
               2.0107212 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.01679596 = queryNorm
             0.48563117 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.046881825 = weight(abstract_txt:system in 4035) [ClassicSimilarity], result of:
           0.046881825 = score(doc=4035,freq=3.0), product of:
             0.12842116 = queryWeight, product of:
               2.2672763 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.01679596 = queryNorm
             0.3650631 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.056643493 = weight(abstract_txt:systems in 4035) [ClassicSimilarity], result of:
           0.056643493 = score(doc=4035,freq=3.0), product of:
             0.15336134 = queryWeight, product of:
               2.6761935 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.01679596 = queryNorm
             0.36934665 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.32120258 = weight(abstract_txt:components in 4035) [ClassicSimilarity], result of:
           0.32120258 = score(doc=4035,freq=5.0), product of:
             0.41132545 = queryWeight, product of:
               4.3828073 = boost
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.01679596 = queryNorm
             0.7808964 = fieldWeight in 4035, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
       0.36 = coord(9/25)
   

2. Greengrass, M.: Conflation methods for searching databases of Latin text (1996) 0.12

   0.12357528 = sum of:
     0.12357528 = product of:
       0.514897 = sum of:
         0.13112167 = weight(abstract_txt:stemmers in 6987) [ClassicSimilarity], result of:
           0.13112167 = score(doc=6987,freq=1.0), product of:
             0.15441129 = queryWeight, product of:
               1.0149627 = boost
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.01679596 = queryNorm
             0.8491715 = fieldWeight in 6987, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.05783 = idf(docFreq=13, maxDocs=44218)
               0.09375 = fieldNorm(doc=6987)
         0.03141629 = weight(abstract_txt:retrieval in 6987) [ClassicSimilarity], result of:
           0.03141629 = score(doc=6987,freq=2.0), product of:
             0.068186186 = queryWeight, product of:
               1.1682059 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.01679596 = queryNorm
             0.4607427 = fieldWeight in 6987, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.09375 = fieldNorm(doc=6987)
         0.017512325 = weight(abstract_txt:which in 6987) [ClassicSimilarity], result of:
           0.017512325 = score(doc=6987,freq=1.0), product of:
             0.06404402 = queryWeight, product of:
               1.3073138 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.01679596 = queryNorm
             0.273442 = fieldWeight in 6987, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.09375 = fieldNorm(doc=6987)
         0.070322745 = weight(abstract_txt:system in 6987) [ClassicSimilarity], result of:
           0.070322745 = score(doc=6987,freq=3.0), product of:
             0.12842116 = queryWeight, product of:
               2.2672763 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.01679596 = queryNorm
             0.54759467 = fieldWeight in 6987, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.09375 = fieldNorm(doc=6987)
         0.04905471 = weight(abstract_txt:systems in 6987) [ClassicSimilarity], result of:
           0.04905471 = score(doc=6987,freq=1.0), product of:
             0.15336134 = queryWeight, product of:
               2.6761935 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.01679596 = queryNorm
             0.3198636 = fieldWeight in 6987, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.09375 = fieldNorm(doc=6987)
         0.21546924 = weight(abstract_txt:components in 6987) [ClassicSimilarity], result of:
           0.21546924 = score(doc=6987,freq=1.0), product of:
             0.41132545 = queryWeight, product of:
               4.3828073 = boost
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.01679596 = queryNorm
             0.52384126 = fieldWeight in 6987, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.09375 = fieldNorm(doc=6987)
       0.24 = coord(6/25)
   

3. Progress in visual information access and retrieval (1999) 0.12

   0.11546865 = sum of:
     0.11546865 = product of:
       0.32074624 = sum of:
         0.017020637 = weight(abstract_txt:since in 839) [ClassicSimilarity], result of:
           0.017020637 = score(doc=839,freq=2.0), product of:
             0.090009585 = queryWeight, product of:
               1.0958974 = boost
               4.890058 = idf(docFreq=903, maxDocs=44218)
               0.01679596 = queryNorm
             0.18909806 = fieldWeight in 839, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.890058 = idf(docFreq=903, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.031741858 = weight(abstract_txt:retrieval in 839) [ClassicSimilarity], result of:
           0.031741858 = score(doc=839,freq=24.0), product of:
             0.068186186 = queryWeight, product of:
               1.1682059 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.01679596 = queryNorm
             0.46551743 = fieldWeight in 839, product of:
               4.8989797 = tf(freq=24.0), with freq of:
                 24.0 = termFreq=24.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.083179414 = weight(abstract_txt:visual in 839) [ClassicSimilarity], result of:
           0.083179414 = score(doc=839,freq=21.0), product of:
             0.11837291 = queryWeight, product of:
               1.2567582 = boost
               5.6078424 = idf(docFreq=440, maxDocs=44218)
               0.01679596 = queryNorm
             0.7026896 = fieldWeight in 839, product of:
               4.582576 = tf(freq=21.0), with freq of:
                 21.0 = termFreq=21.0
               5.6078424 = idf(docFreq=440, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.011421301 = weight(abstract_txt:which in 839) [ClassicSimilarity], result of:
           0.011421301 = score(doc=839,freq=5.0), product of:
             0.06404402 = queryWeight, product of:
               1.3073138 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.01679596 = queryNorm
             0.17833517 = fieldWeight in 839, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.02237995 = weight(abstract_txt:contributions in 839) [ClassicSimilarity], result of:
           0.02237995 = score(doc=839,freq=1.0), product of:
             0.13610923 = queryWeight, product of:
               1.3476259 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.01679596 = queryNorm
             0.16442639 = fieldWeight in 839, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.014521163 = weight(abstract_txt:several in 839) [ClassicSimilarity], result of:
           0.014521163 = score(doc=839,freq=1.0), product of:
             0.116774015 = queryWeight, product of:
               1.5287775 = boost
               4.5477557 = idf(docFreq=1272, maxDocs=44218)
               0.01679596 = queryNorm
             0.12435269 = fieldWeight in 839, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.5477557 = idf(docFreq=1272, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.037168633 = weight(abstract_txt:evaluation in 839) [ClassicSimilarity], result of:
           0.037168633 = score(doc=839,freq=4.0), product of:
             0.15150347 = queryWeight, product of:
               2.0107212 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.01679596 = queryNorm
             0.24533188 = fieldWeight in 839, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.040468067 = weight(abstract_txt:systems in 839) [ClassicSimilarity], result of:
           0.040468067 = score(doc=839,freq=8.0), product of:
             0.15336134 = queryWeight, product of:
               2.6761935 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.01679596 = queryNorm
             0.263874 = fieldWeight in 839, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
         0.06284519 = weight(abstract_txt:components in 839) [ClassicSimilarity], result of:
           0.06284519 = score(doc=839,freq=1.0), product of:
             0.41132545 = queryWeight, product of:
               4.3828073 = boost
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.01679596 = queryNorm
             0.15278703 = fieldWeight in 839, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.02734375 = fieldNorm(doc=839)
       0.36 = coord(9/25)
   

4. Tague-Sutcliffe, J.M.: Some perspectives on the evaluation of information retrieval systems (1996) 0.11

   0.11379756 = sum of:
     0.11379756 = product of:
       0.4741565 = sum of:
         0.034918502 = weight(abstract_txt:possible in 4163) [ClassicSimilarity], result of:
           0.034918502 = score(doc=4163,freq=1.0), product of:
             0.0805273 = queryWeight, product of:
               1.0365664 = boost
               4.6253138 = idf(docFreq=1177, maxDocs=44218)
               0.01679596 = queryNorm
             0.43362316 = fieldWeight in 4163, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6253138 = idf(docFreq=1177, maxDocs=44218)
               0.09375 = fieldNorm(doc=4163)
         0.044429343 = weight(abstract_txt:retrieval in 4163) [ClassicSimilarity], result of:
           0.044429343 = score(doc=4163,freq=4.0), product of:
             0.068186186 = queryWeight, product of:
               1.1682059 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.01679596 = queryNorm
             0.6515886 = fieldWeight in 4163, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.09375 = fieldNorm(doc=4163)
         0.017512325 = weight(abstract_txt:which in 4163) [ClassicSimilarity], result of:
           0.017512325 = score(doc=4163,freq=1.0), product of:
             0.06404402 = queryWeight, product of:
               1.3073138 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.01679596 = queryNorm
             0.273442 = fieldWeight in 4163, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.09375 = fieldNorm(doc=4163)
         0.063717656 = weight(abstract_txt:evaluation in 4163) [ClassicSimilarity], result of:
           0.063717656 = score(doc=4163,freq=1.0), product of:
             0.15150347 = queryWeight, product of:
               2.0107212 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.01679596 = queryNorm
             0.42056894 = fieldWeight in 4163, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=4163)
         0.09810942 = weight(abstract_txt:systems in 4163) [ClassicSimilarity], result of:
           0.09810942 = score(doc=4163,freq=4.0), product of:
             0.15336134 = queryWeight, product of:
               2.6761935 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.01679596 = queryNorm
             0.6397272 = fieldWeight in 4163, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.09375 = fieldNorm(doc=4163)
         0.21546924 = weight(abstract_txt:components in 4163) [ClassicSimilarity], result of:
           0.21546924 = score(doc=4163,freq=1.0), product of:
             0.41132545 = queryWeight, product of:
               4.3828073 = boost
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.01679596 = queryNorm
             0.52384126 = fieldWeight in 4163, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.09375 = fieldNorm(doc=4163)
       0.24 = coord(6/25)
   

5. Vanderwende, L.; Suzuki, H.; Brockett, J.M.; Nenkova, A.: Beyond SumBasic : task-focused summarization with sentence simplification and lexical expansion (2007) 0.11

   0.11250615 = sum of:
     0.11250615 = product of:
       0.46877563 = sum of:
         0.034126908 = weight(abstract_txt:together in 948) [ClassicSimilarity], result of:
           0.034126908 = score(doc=948,freq=1.0), product of:
             0.103919774 = queryWeight, product of:
               1.1775372 = boost
               5.254347 = idf(docFreq=627, maxDocs=44218)
               0.01679596 = queryNorm
             0.32839668 = fieldWeight in 948, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.254347 = idf(docFreq=627, maxDocs=44218)
               0.0625 = fieldNorm(doc=948)
         0.051154166 = weight(abstract_txt:contributions in 948) [ClassicSimilarity], result of:
           0.051154166 = score(doc=948,freq=1.0), product of:
             0.13610923 = queryWeight, product of:
               1.3476259 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.01679596 = queryNorm
             0.37583172 = fieldWeight in 948, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0625 = fieldNorm(doc=948)
         0.07357481 = weight(abstract_txt:evaluation in 948) [ClassicSimilarity], result of:
           0.07357481 = score(doc=948,freq=3.0), product of:
             0.15150347 = queryWeight, product of:
               2.0107212 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.01679596 = queryNorm
             0.48563117 = fieldWeight in 948, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=948)
         0.060524177 = weight(abstract_txt:system in 948) [ClassicSimilarity], result of:
           0.060524177 = score(doc=948,freq=5.0), product of:
             0.12842116 = queryWeight, product of:
               2.2672763 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.01679596 = queryNorm
             0.47129443 = fieldWeight in 948, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=948)
         0.046249222 = weight(abstract_txt:systems in 948) [ClassicSimilarity], result of:
           0.046249222 = score(doc=948,freq=2.0), product of:
             0.15336134 = queryWeight, product of:
               2.6761935 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.01679596 = queryNorm
             0.3015703 = fieldWeight in 948, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=948)
         0.20314634 = weight(abstract_txt:components in 948) [ClassicSimilarity], result of:
           0.20314634 = score(doc=948,freq=2.0), product of:
             0.41132545 = queryWeight, product of:
               4.3828073 = boost
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.01679596 = queryNorm
             0.49388224 = fieldWeight in 948, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.58764 = idf(docFreq=449, maxDocs=44218)
               0.0625 = fieldNorm(doc=948)
       0.24 = coord(6/25)