Document (#42051)

Author
Angelini, M.
Fazzini, V.
Ferro, N.
Santucci, G.
Silvello, G.
Title
CLAIRE: A combinatorial visual analytics system for information retrieval evaluation
Source
Information processing and management. 54(2018) no.6, S.1077-1100
Year
2018
Abstract
Information Retrieval (IR) develops complex systems, constituted of several components, which aim at returning and optimally ranking the most relevant documents in response to user queries. In this context, experimental evaluation plays a central role, since it allows for measuring IR systems effectiveness, increasing the understanding of their functioning, and better directing the efforts for improving them. Current evaluation methodologies are limited by two major factors: (i) IR systems are evaluated as "black boxes", since it is not possible to decompose the contributions of the different components, e.g., stop lists, stemmers, and IR models; (ii) given that it is not possible to predict the effectiveness of an IR system, both academia and industry need to explore huge numbers of systems, originated by large combinatorial compositions of their components, to understand how they perform and how these components interact together. We propose a Combinatorial visuaL Analytics system for Information Retrieval Evaluation (CLAIRE) which allows for exploring and making sense of the performances of a large amount of IR systems, in order to quickly and intuitively grasp which system configurations are preferred, what are the contributions of the different components and how these components interact together. The CLAIRE system is then validated against use cases based on several test collections using a wide set of systems, generated by a combinatorial composition of several off-the-shelf components, representing the most common denominator almost always present in English IR systems. In particular, we validate the findings enabled by CLAIRE with respect to consolidated deep statistical analyses and we show that the CLAIRE system allows the generation of new insights, which were not detectable with traditional approaches.
Content
Vgl.: https://doi.org/10.1016/j.ipm.2018.04.006.
Theme
Retrievalstudien

Similar documents (author)

  1. Ferro, N.; Silvello, G.: NESTOR: a formal model for digital archives (2013) 6.87
    6.870943 = sum of:
      6.870943 = sum of:
        3.4354715 = weight(author_txt:silvello in 4708) [ClassicSimilarity], result of:
          3.4354715 = score(doc=4708,freq=1.0), product of:
            0.7071068 = queryWeight, product of:
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.07277022 = queryNorm
            4.85849 = fieldWeight in 4708, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.5 = fieldNorm(doc=4708)
        3.4354715 = weight(author_txt:ferro in 4708) [ClassicSimilarity], result of:
          3.4354715 = score(doc=4708,freq=1.0), product of:
            0.7071068 = queryWeight, product of:
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.07277022 = queryNorm
            4.85849 = fieldWeight in 4708, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.5 = fieldNorm(doc=4708)
    
  2. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 6.87
    6.870943 = sum of:
      6.870943 = sum of:
        3.4354715 = weight(author_txt:silvello in 36) [ClassicSimilarity], result of:
          3.4354715 = score(doc=36,freq=1.0), product of:
            0.7071068 = queryWeight, product of:
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.07277022 = queryNorm
            4.85849 = fieldWeight in 36, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.5 = fieldNorm(doc=36)
        3.4354715 = weight(author_txt:ferro in 36) [ClassicSimilarity], result of:
          3.4354715 = score(doc=36,freq=1.0), product of:
            0.7071068 = queryWeight, product of:
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.07277022 = queryNorm
            4.85849 = fieldWeight in 36, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.5 = fieldNorm(doc=36)
    
  3. Ferro, N.; Silvello, G.; Keskustalo, H.; Pirkola, A.; Järvelin, K.: ¬The twist measure for IR evaluation : taking user's effort into account (2016) 4.29
    4.294339 = sum of:
      4.294339 = sum of:
        2.1471696 = weight(author_txt:silvello in 4772) [ClassicSimilarity], result of:
          2.1471696 = score(doc=4772,freq=1.0), product of:
            0.7071068 = queryWeight, product of:
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.07277022 = queryNorm
            3.0365562 = fieldWeight in 4772, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.3125 = fieldNorm(doc=4772)
        2.1471696 = weight(author_txt:ferro in 4772) [ClassicSimilarity], result of:
          2.1471696 = score(doc=4772,freq=1.0), product of:
            0.7071068 = queryWeight, product of:
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.07277022 = queryNorm
            3.0365562 = fieldWeight in 4772, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.71698 = idf(docFreq=6, maxDocs=42740)
              0.3125 = fieldNorm(doc=4772)
    
  4. Silvello, G.: Learning to cite framework : how to automatically construct citations for hierarchical data (2017) 2.15
    2.1471696 = sum of:
      2.1471696 = product of:
        4.294339 = sum of:
          4.294339 = weight(author_txt:silvello in 5648) [ClassicSimilarity], result of:
            4.294339 = score(doc=5648,freq=1.0), product of:
              0.7071068 = queryWeight, product of:
                9.71698 = idf(docFreq=6, maxDocs=42740)
                0.07277022 = queryNorm
              6.0731125 = fieldWeight in 5648, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.71698 = idf(docFreq=6, maxDocs=42740)
                0.625 = fieldNorm(doc=5648)
        0.5 = coord(1/2)
    
  5. Silvello, G.: Theory and practice of data citation (2018) 2.15
    2.1471696 = sum of:
      2.1471696 = product of:
        4.294339 = sum of:
          4.294339 = weight(author_txt:silvello in 7) [ClassicSimilarity], result of:
            4.294339 = score(doc=7,freq=1.0), product of:
              0.7071068 = queryWeight, product of:
                9.71698 = idf(docFreq=6, maxDocs=42740)
                0.07277022 = queryNorm
              6.0731125 = fieldWeight in 7, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.71698 = idf(docFreq=6, maxDocs=42740)
                0.625 = fieldNorm(doc=7)
        0.5 = coord(1/2)
    

Similar documents (content)

  1. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 0.27
    0.26686975 = sum of:
      0.26686975 = product of:
        0.7413049 = sum of:
          0.085815564 = weight(abstract_txt:stemmers in 36) [ClassicSimilarity], result of:
            0.085815564 = score(doc=36,freq=1.0), product of:
              0.15215807 = queryWeight, product of:
                1.0077045 = boost
                9.023833 = idf(docFreq=13, maxDocs=42740)
                0.01673288 = queryNorm
              0.5639896 = fieldWeight in 36, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.023833 = idf(docFreq=13, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.02060934 = weight(abstract_txt:retrieval in 36) [ClassicSimilarity], result of:
            0.02060934 = score(doc=36,freq=2.0), product of:
              0.06729623 = queryWeight, product of:
                1.160758 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.01673288 = queryNorm
              0.30624807 = fieldWeight in 36, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.05913288 = weight(abstract_txt:together in 36) [ClassicSimilarity], result of:
            0.05913288 = score(doc=36,freq=3.0), product of:
              0.10369868 = queryWeight, product of:
                1.1764876 = boost
                5.267629 = idf(docFreq=598, maxDocs=42740)
                0.01673288 = queryNorm
              0.5702376 = fieldWeight in 36, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.267629 = idf(docFreq=598, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.011792564 = weight(abstract_txt:which in 36) [ClassicSimilarity], result of:
            0.011792564 = score(doc=36,freq=1.0), product of:
              0.064319566 = queryWeight, product of:
                1.3103498 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.01673288 = queryNorm
              0.18334334 = fieldWeight in 36, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.06834911 = weight(abstract_txt:interact in 36) [ClassicSimilarity], result of:
            0.06834911 = score(doc=36,freq=1.0), product of:
              0.1647212 = queryWeight, product of:
                1.4827757 = boost
                6.63901 = idf(docFreq=151, maxDocs=42740)
                0.01673288 = queryNorm
              0.41493812 = fieldWeight in 36, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.63901 = idf(docFreq=151, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.073376395 = weight(abstract_txt:evaluation in 36) [ClassicSimilarity], result of:
            0.073376395 = score(doc=36,freq=3.0), product of:
              0.15086949 = queryWeight, product of:
                2.006857 = boost
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.01673288 = queryNorm
              0.48635674 = fieldWeight in 36, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.046176735 = weight(abstract_txt:system in 36) [ClassicSimilarity], result of:
            0.046176735 = score(doc=36,freq=3.0), product of:
              0.12682642 = queryWeight, product of:
                2.2535448 = boost
                3.3633559 = idf(docFreq=4021, maxDocs=42740)
                0.01673288 = queryNorm
              0.36409396 = fieldWeight in 36, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.3633559 = idf(docFreq=4021, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.05637412 = weight(abstract_txt:systems in 36) [ClassicSimilarity], result of:
            0.05637412 = score(doc=36,freq=3.0), product of:
              0.15250932 = queryWeight, product of:
                2.6692111 = boost
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.01673288 = queryNorm
              0.36964378 = fieldWeight in 36, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
          0.3196782 = weight(abstract_txt:components in 36) [ClassicSimilarity], result of:
            0.3196782 = score(doc=36,freq=5.0), product of:
              0.40904298 = queryWeight, product of:
                4.3713865 = boost
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.01673288 = queryNorm
              0.7815271 = fieldWeight in 36, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.0625 = fieldNorm(doc=36)
        0.36 = coord(9/25)
    
  2. Greengrass, M.: Conflation methods for searching databases of Latin text (1996) 0.12
    0.12236625 = sum of:
      0.12236625 = product of:
        0.5098594 = sum of:
          0.12872334 = weight(abstract_txt:stemmers in 57) [ClassicSimilarity], result of:
            0.12872334 = score(doc=57,freq=1.0), product of:
              0.15215807 = queryWeight, product of:
                1.0077045 = boost
                9.023833 = idf(docFreq=13, maxDocs=42740)
                0.01673288 = queryNorm
              0.84598434 = fieldWeight in 57, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.023833 = idf(docFreq=13, maxDocs=42740)
                0.09375 = fieldNorm(doc=57)
          0.03091401 = weight(abstract_txt:retrieval in 57) [ClassicSimilarity], result of:
            0.03091401 = score(doc=57,freq=2.0), product of:
              0.06729623 = queryWeight, product of:
                1.160758 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.01673288 = queryNorm
              0.4593721 = fieldWeight in 57, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.09375 = fieldNorm(doc=57)
          0.017688844 = weight(abstract_txt:which in 57) [ClassicSimilarity], result of:
            0.017688844 = score(doc=57,freq=1.0), product of:
              0.064319566 = queryWeight, product of:
                1.3103498 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.01673288 = queryNorm
              0.275015 = fieldWeight in 57, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.09375 = fieldNorm(doc=57)
          0.0692651 = weight(abstract_txt:system in 57) [ClassicSimilarity], result of:
            0.0692651 = score(doc=57,freq=3.0), product of:
              0.12682642 = queryWeight, product of:
                2.2535448 = boost
                3.3633559 = idf(docFreq=4021, maxDocs=42740)
                0.01673288 = queryNorm
              0.5461409 = fieldWeight in 57, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.3633559 = idf(docFreq=4021, maxDocs=42740)
                0.09375 = fieldNorm(doc=57)
          0.04882142 = weight(abstract_txt:systems in 57) [ClassicSimilarity], result of:
            0.04882142 = score(doc=57,freq=1.0), product of:
              0.15250932 = queryWeight, product of:
                2.6692111 = boost
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.01673288 = queryNorm
              0.3201209 = fieldWeight in 57, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.09375 = fieldNorm(doc=57)
          0.21444665 = weight(abstract_txt:components in 57) [ClassicSimilarity], result of:
            0.21444665 = score(doc=57,freq=1.0), product of:
              0.40904298 = queryWeight, product of:
                4.3713865 = boost
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.01673288 = queryNorm
              0.52426434 = fieldWeight in 57, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.09375 = fieldNorm(doc=57)
        0.24 = coord(6/25)
    
  3. Progress in visual information access and retrieval (1999) 0.12
    0.115421765 = sum of:
      0.115421765 = product of:
        0.320616 = sum of:
          0.017040031 = weight(abstract_txt:since in 1965) [ClassicSimilarity], result of:
            0.017040031 = score(doc=1965,freq=2.0), product of:
              0.08986264 = queryWeight, product of:
                1.0951921 = boost
                4.903635 = idf(docFreq=861, maxDocs=42740)
                0.01673288 = queryNorm
              0.18962309 = fieldWeight in 1965, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.903635 = idf(docFreq=861, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.031234372 = weight(abstract_txt:retrieval in 1965) [ClassicSimilarity], result of:
            0.031234372 = score(doc=1965,freq=24.0), product of:
              0.06729623 = queryWeight, product of:
                1.160758 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.01673288 = queryNorm
              0.46413258 = fieldWeight in 1965, product of:
                4.8989797 = tf(freq=24.0), with freq of:
                  24.0 = termFreq=24.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.083558455 = weight(abstract_txt:visual in 1965) [ClassicSimilarity], result of:
            0.083558455 = score(doc=1965,freq=21.0), product of:
              0.11844846 = queryWeight, product of:
                1.2573767 = boost
                5.629804 = idf(docFreq=416, maxDocs=42740)
                0.01673288 = queryNorm
              0.7054415 = fieldWeight in 1965, product of:
                4.582576 = tf(freq=21.0), with freq of:
                  21.0 = termFreq=21.0
                5.629804 = idf(docFreq=416, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.011536427 = weight(abstract_txt:which in 1965) [ClassicSimilarity], result of:
            0.011536427 = score(doc=1965,freq=5.0), product of:
              0.064319566 = queryWeight, product of:
                1.3103498 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.01673288 = queryNorm
              0.17936108 = fieldWeight in 1965, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.02283369 = weight(abstract_txt:contributions in 1965) [ClassicSimilarity], result of:
            0.02283369 = score(doc=1965,freq=1.0), product of:
              0.13761306 = queryWeight, product of:
                1.3552847 = boost
                6.068179 = idf(docFreq=268, maxDocs=42740)
                0.01673288 = queryNorm
              0.16592677 = fieldWeight in 1965, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.068179 = idf(docFreq=268, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.014522069 = weight(abstract_txt:several in 1965) [ClassicSimilarity], result of:
            0.014522069 = score(doc=1965,freq=1.0), product of:
              0.11649975 = queryWeight, product of:
                1.5272454 = boost
                4.5587463 = idf(docFreq=1216, maxDocs=42740)
                0.01673288 = queryNorm
              0.12465322 = fieldWeight in 1965, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.5587463 = idf(docFreq=1216, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.037068397 = weight(abstract_txt:evaluation in 1965) [ClassicSimilarity], result of:
            0.037068397 = score(doc=1965,freq=4.0), product of:
              0.15086949 = queryWeight, product of:
                2.006857 = boost
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.01673288 = queryNorm
              0.24569842 = fieldWeight in 1965, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.04027562 = weight(abstract_txt:systems in 1965) [ClassicSimilarity], result of:
            0.04027562 = score(doc=1965,freq=8.0), product of:
              0.15250932 = queryWeight, product of:
                2.6692111 = boost
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.01673288 = queryNorm
              0.26408628 = fieldWeight in 1965, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
          0.06254694 = weight(abstract_txt:components in 1965) [ClassicSimilarity], result of:
            0.06254694 = score(doc=1965,freq=1.0), product of:
              0.40904298 = queryWeight, product of:
                4.3713865 = boost
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.01673288 = queryNorm
              0.15291043 = fieldWeight in 1965, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.02734375 = fieldNorm(doc=1965)
        0.36 = coord(9/25)
    
  4. Tague-Sutcliffe, J.M.: Some perspectives on the evaluation of information retrieval systems (1996) 0.11
    0.11325675 = sum of:
      0.11325675 = product of:
        0.47190315 = sum of:
          0.034859955 = weight(abstract_txt:possible in 4232) [ClassicSimilarity], result of:
            0.034859955 = score(doc=4232,freq=1.0), product of:
              0.080244996 = queryWeight, product of:
                1.0349269 = boost
                4.633803 = idf(docFreq=1128, maxDocs=42740)
                0.01673288 = queryNorm
              0.43441904 = fieldWeight in 4232, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.633803 = idf(docFreq=1128, maxDocs=42740)
                0.09375 = fieldNorm(doc=4232)
          0.04371901 = weight(abstract_txt:retrieval in 4232) [ClassicSimilarity], result of:
            0.04371901 = score(doc=4232,freq=4.0), product of:
              0.06729623 = queryWeight, product of:
                1.160758 = boost
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.01673288 = queryNorm
              0.6496502 = fieldWeight in 4232, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.4648013 = idf(docFreq=3633, maxDocs=42740)
                0.09375 = fieldNorm(doc=4232)
          0.017688844 = weight(abstract_txt:which in 4232) [ClassicSimilarity], result of:
            0.017688844 = score(doc=4232,freq=1.0), product of:
              0.064319566 = queryWeight, product of:
                1.3103498 = boost
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.01673288 = queryNorm
              0.275015 = fieldWeight in 4232, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.9334934 = idf(docFreq=6181, maxDocs=42740)
                0.09375 = fieldNorm(doc=4232)
          0.06354582 = weight(abstract_txt:evaluation in 4232) [ClassicSimilarity], result of:
            0.06354582 = score(doc=4232,freq=1.0), product of:
              0.15086949 = queryWeight, product of:
                2.006857 = boost
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.01673288 = queryNorm
              0.4211973 = fieldWeight in 4232, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.09375 = fieldNorm(doc=4232)
          0.09764284 = weight(abstract_txt:systems in 4232) [ClassicSimilarity], result of:
            0.09764284 = score(doc=4232,freq=4.0), product of:
              0.15250932 = queryWeight, product of:
                2.6692111 = boost
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.01673288 = queryNorm
              0.6402418 = fieldWeight in 4232, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.09375 = fieldNorm(doc=4232)
          0.21444665 = weight(abstract_txt:components in 4232) [ClassicSimilarity], result of:
            0.21444665 = score(doc=4232,freq=1.0), product of:
              0.40904298 = queryWeight, product of:
                4.3713865 = boost
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.01673288 = queryNorm
              0.52426434 = fieldWeight in 4232, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.09375 = fieldNorm(doc=4232)
        0.24 = coord(6/25)
    
  5. Vanderwende, L.; Suzuki, H.; Brockett, J.M.; Nenkova, A.: Beyond SumBasic : task-focused summarization with sentence simplification and lexical expansion (2007) 0.11
    0.11220803 = sum of:
      0.11220803 = product of:
        0.46753347 = sum of:
          0.034140386 = weight(abstract_txt:together in 2949) [ClassicSimilarity], result of:
            0.034140386 = score(doc=2949,freq=1.0), product of:
              0.10369868 = queryWeight, product of:
                1.1764876 = boost
                5.267629 = idf(docFreq=598, maxDocs=42740)
                0.01673288 = queryNorm
              0.32922682 = fieldWeight in 2949, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.267629 = idf(docFreq=598, maxDocs=42740)
                0.0625 = fieldNorm(doc=2949)
          0.052191295 = weight(abstract_txt:contributions in 2949) [ClassicSimilarity], result of:
            0.052191295 = score(doc=2949,freq=1.0), product of:
              0.13761306 = queryWeight, product of:
                1.3552847 = boost
                6.068179 = idf(docFreq=268, maxDocs=42740)
                0.01673288 = queryNorm
              0.3792612 = fieldWeight in 2949, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.068179 = idf(docFreq=268, maxDocs=42740)
                0.0625 = fieldNorm(doc=2949)
          0.073376395 = weight(abstract_txt:evaluation in 2949) [ClassicSimilarity], result of:
            0.073376395 = score(doc=2949,freq=3.0), product of:
              0.15086949 = queryWeight, product of:
                2.006857 = boost
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.01673288 = queryNorm
              0.48635674 = fieldWeight in 2949, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.492771 = idf(docFreq=1299, maxDocs=42740)
                0.0625 = fieldNorm(doc=2949)
          0.059613906 = weight(abstract_txt:system in 2949) [ClassicSimilarity], result of:
            0.059613906 = score(doc=2949,freq=5.0), product of:
              0.12682642 = queryWeight, product of:
                2.2535448 = boost
                3.3633559 = idf(docFreq=4021, maxDocs=42740)
                0.01673288 = queryNorm
              0.47004327 = fieldWeight in 2949, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.3633559 = idf(docFreq=4021, maxDocs=42740)
                0.0625 = fieldNorm(doc=2949)
          0.046029277 = weight(abstract_txt:systems in 2949) [ClassicSimilarity], result of:
            0.046029277 = score(doc=2949,freq=2.0), product of:
              0.15250932 = queryWeight, product of:
                2.6692111 = boost
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.01673288 = queryNorm
              0.3018129 = fieldWeight in 2949, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.414623 = idf(docFreq=3820, maxDocs=42740)
                0.0625 = fieldNorm(doc=2949)
          0.20218222 = weight(abstract_txt:components in 2949) [ClassicSimilarity], result of:
            0.20218222 = score(doc=2949,freq=2.0), product of:
              0.40904298 = queryWeight, product of:
                4.3713865 = boost
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.01673288 = queryNorm
              0.4942811 = fieldWeight in 2949, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5921526 = idf(docFreq=432, maxDocs=42740)
                0.0625 = fieldNorm(doc=2949)
        0.24 = coord(6/25)