Document (#32965)

Author
Tsai, C.-F.
McGarry, K.
Tait, J.
Title
Qualitative evaluation of automatic assignment of keywords to images
Source
Information processing and management. 42(2006) no.1, S.136-154
Year
2006
Abstract
In image retrieval, most systems lack user-centred evaluation since they are assessed by some chosen ground truth dataset. The results reported through precision and recall assessed against the ground truth are thought of as being an acceptable surrogate for the judgment of real users. Much current research focuses on automatically assigning keywords to images for enhancing retrieval effectiveness. However, evaluation methods are usually based on system-level assessment, e.g. classification accuracy based on some chosen ground truth dataset. In this paper, we present a qualitative evaluation methodology for automatic image indexing systems. The automatic indexing task is formulated as one of image annotation, or automatic metadata generation for images. The evaluation is composed of two individual methods. First, the automatic indexing annotation results are assessed by human subjects. Second, the subjects are asked to annotate some chosen images as the test set whose annotations are used as ground truth. Then, the system is tested by the test set whose annotation results are judged against the ground truth. Only one of these methods is reported for most systems on which user-centred evaluation are conducted. We believe that both methods need to be considered for full evaluation. We also provide an example evaluation of our system based on this methodology. According to this study, our proposed evaluation methodology is able to provide deeper understanding of the system's performance.
Theme
Automatisches Indexieren
Form
Bilder

Similar documents (author)

  1. Tait, J.: CALS and its implications for the library and information retrieval communities (1994) 1.21
    1.2070096 = sum of:
      1.2070096 = product of:
        3.6210287 = sum of:
          3.6210287 = weight(author_txt:tait in 2730) [ClassicSimilarity], result of:
            3.6210287 = score(doc=2730,freq=1.0), product of:
              0.61816955 = queryWeight, product of:
                1.0735195 = boost
                9.37226 = idf(docFreq=9, maxDocs=43254)
                0.0614403 = queryNorm
              5.8576627 = fieldWeight in 2730, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.37226 = idf(docFreq=9, maxDocs=43254)
                0.625 = fieldNorm(doc=2730)
        0.33333334 = coord(1/3)
    
  2. Tait, J.A.: Authors and titles : an analytical study of the author concept in codes of cataloguing rules in the English language, from that of the British Museum in 1841 to the Anglo-American Cataloguing Rules 1967 (1969) 1.21
    1.2070096 = sum of:
      1.2070096 = product of:
        3.6210287 = sum of:
          3.6210287 = weight(author_txt:tait in 4162) [ClassicSimilarity], result of:
            3.6210287 = score(doc=4162,freq=1.0), product of:
              0.61816955 = queryWeight, product of:
                1.0735195 = boost
                9.37226 = idf(docFreq=9, maxDocs=43254)
                0.0614403 = queryNorm
              5.8576627 = fieldWeight in 4162, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.37226 = idf(docFreq=9, maxDocs=43254)
                0.625 = fieldNorm(doc=4162)
        0.33333334 = coord(1/3)
    
  3. McGarry, D.: Report from the section on classification and indexing : 1989-90 (1990) 0.98
    0.9756207 = sum of:
      0.9756207 = product of:
        2.926862 = sum of:
          2.926862 = weight(author_txt:mcgarry in 475) [ClassicSimilarity], result of:
            2.926862 = score(doc=475,freq=1.0), product of:
              0.53639877 = queryWeight, product of:
                8.730406 = idf(docFreq=18, maxDocs=43254)
                0.0614403 = queryNorm
              5.456504 = fieldWeight in 475, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.730406 = idf(docFreq=18, maxDocs=43254)
                0.625 = fieldNorm(doc=475)
        0.33333334 = coord(1/3)
    
  4. McGarry, D.: Displays of bibliographic records in call number order : functions of the displays and data elements needed (1992) 0.98
    0.9756207 = sum of:
      0.9756207 = product of:
        2.926862 = sum of:
          2.926862 = weight(author_txt:mcgarry in 2384) [ClassicSimilarity], result of:
            2.926862 = score(doc=2384,freq=1.0), product of:
              0.53639877 = queryWeight, product of:
                8.730406 = idf(docFreq=18, maxDocs=43254)
                0.0614403 = queryNorm
              5.456504 = fieldWeight in 2384, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.730406 = idf(docFreq=18, maxDocs=43254)
                0.625 = fieldNorm(doc=2384)
        0.33333334 = coord(1/3)
    
  5. McGarry, D.: Report from the section on classification and indexing : 1991-1992 (1992) 0.98
    0.9756207 = sum of:
      0.9756207 = product of:
        2.926862 = sum of:
          2.926862 = weight(author_txt:mcgarry in 2608) [ClassicSimilarity], result of:
            2.926862 = score(doc=2608,freq=1.0), product of:
              0.53639877 = queryWeight, product of:
                8.730406 = idf(docFreq=18, maxDocs=43254)
                0.0614403 = queryNorm
              5.456504 = fieldWeight in 2608, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.730406 = idf(docFreq=18, maxDocs=43254)
                0.625 = fieldNorm(doc=2608)
        0.33333334 = coord(1/3)
    

Similar documents (content)

  1. Ostermann, F.O.; Tomko, M.; Purves, R.: User evaluation of automatically generated keywords and toponyms for geo-referenced images (2013) 0.34
    0.33564106 = sum of:
      0.33564106 = product of:
        0.8391026 = sum of:
          0.019579714 = weight(abstract_txt:results in 2128) [ClassicSimilarity], result of:
            0.019579714 = score(doc=2128,freq=2.0), product of:
              0.06327844 = queryWeight, product of:
                1.0404288 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.017373519 = queryNorm
              0.3094216 = fieldWeight in 2128, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.016107967 = weight(abstract_txt:some in 2128) [ClassicSimilarity], result of:
            0.016107967 = score(doc=2128,freq=1.0), product of:
              0.06999849 = queryWeight, product of:
                1.0942808 = boost
                3.6819005 = idf(docFreq=2959, maxDocs=43254)
                0.017373519 = queryNorm
              0.23011878 = fieldWeight in 2128, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6819005 = idf(docFreq=2959, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.0530279 = weight(abstract_txt:qualitative in 2128) [ClassicSimilarity], result of:
            0.0530279 = score(doc=2128,freq=2.0), product of:
              0.10740489 = queryWeight, product of:
                1.1067533 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.017373519 = queryNorm
              0.49371964 = fieldWeight in 2128, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.09498555 = weight(abstract_txt:keywords in 2128) [ClassicSimilarity], result of:
            0.09498555 = score(doc=2128,freq=4.0), product of:
              0.12573305 = queryWeight, product of:
                1.197467 = boost
                6.043633 = idf(docFreq=278, maxDocs=43254)
                0.017373519 = queryNorm
              0.7554541 = fieldWeight in 2128, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.043633 = idf(docFreq=278, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.0867546 = weight(abstract_txt:image in 2128) [ClassicSimilarity], result of:
            0.0867546 = score(doc=2128,freq=3.0), product of:
              0.14912464 = queryWeight, product of:
                1.5971994 = boost
                5.374059 = idf(docFreq=544, maxDocs=43254)
                0.017373519 = queryNorm
              0.581759 = fieldWeight in 2128, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.374059 = idf(docFreq=544, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.043982785 = weight(abstract_txt:methods in 2128) [ClassicSimilarity], result of:
            0.043982785 = score(doc=2128,freq=2.0), product of:
              0.11945919 = queryWeight, product of:
                1.6506828 = boost
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.017373519 = queryNorm
              0.3681825 = fieldWeight in 2128, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.116017744 = weight(abstract_txt:annotation in 2128) [ClassicSimilarity], result of:
            0.116017744 = score(doc=2128,freq=1.0), product of:
              0.26106247 = queryWeight, product of:
                2.1132784 = boost
                7.110497 = idf(docFreq=95, maxDocs=43254)
                0.017373519 = queryNorm
              0.44440606 = fieldWeight in 2128, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.110497 = idf(docFreq=95, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.18136537 = weight(abstract_txt:images in 2128) [ClassicSimilarity], result of:
            0.18136537 = score(doc=2128,freq=7.0), product of:
              0.20232272 = queryWeight, product of:
                2.1482072 = boost
                5.421016 = idf(docFreq=519, maxDocs=43254)
                0.017373519 = queryNorm
              0.89641625 = fieldWeight in 2128, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                5.421016 = idf(docFreq=519, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.07546834 = weight(abstract_txt:automatic in 2128) [ClassicSimilarity], result of:
            0.07546834 = score(doc=2128,freq=1.0), product of:
              0.23237404 = queryWeight, product of:
                2.573965 = boost
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.017373519 = queryNorm
              0.32477096 = fieldWeight in 2128, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
          0.15181269 = weight(abstract_txt:evaluation in 2128) [ClassicSimilarity], result of:
            0.15181269 = score(doc=2128,freq=3.0), product of:
              0.3123203 = queryWeight, product of:
                4.0035496 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.017373519 = queryNorm
              0.4860801 = fieldWeight in 2128, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=2128)
        0.4 = coord(10/25)
    
  2. Hu, X.; Lee, J.H.; Bainbridge, D.; Choi, K.; Organisciak, P.; Downie, J.S.: ¬The MIREX grand challenge : a framework of holistic user-experience evaluation in music information retrieval (2017) 0.26
    0.2591509 = sum of:
      0.2591509 = product of:
        0.9255389 = sum of:
          0.021730978 = weight(abstract_txt:system in 4786) [ClassicSimilarity], result of:
            0.021730978 = score(doc=4786,freq=2.0), product of:
              0.05845626 = queryWeight, product of:
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.017373519 = queryNorm
              0.37174767 = fieldWeight in 4786, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
          0.035879683 = weight(abstract_txt:systems in 4786) [ClassicSimilarity], result of:
            0.035879683 = score(doc=4786,freq=5.0), product of:
              0.060167767 = queryWeight, product of:
                1.0145335 = boost
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.017373519 = queryNorm
              0.5963273 = fieldWeight in 4786, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
          0.017306186 = weight(abstract_txt:results in 4786) [ClassicSimilarity], result of:
            0.017306186 = score(doc=4786,freq=1.0), product of:
              0.06327844 = queryWeight, product of:
                1.0404288 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.017373519 = queryNorm
              0.27349263 = fieldWeight in 4786, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
          0.05389041 = weight(abstract_txt:against in 4786) [ClassicSimilarity], result of:
            0.05389041 = score(doc=4786,freq=1.0), product of:
              0.11787781 = queryWeight, product of:
                1.1594576 = boost
                5.851799 = idf(docFreq=337, maxDocs=43254)
                0.017373519 = queryNorm
              0.4571718 = fieldWeight in 4786, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.851799 = idf(docFreq=337, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
          0.22989027 = weight(abstract_txt:ground in 4786) [ClassicSimilarity], result of:
            0.22989027 = score(doc=4786,freq=1.0), product of:
              0.4208088 = queryWeight, product of:
                3.4637876 = boost
                6.992714 = idf(docFreq=107, maxDocs=43254)
                0.017373519 = queryNorm
              0.5463058 = fieldWeight in 4786, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.992714 = idf(docFreq=107, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
          0.2769692 = weight(abstract_txt:truth in 4786) [ClassicSimilarity], result of:
            0.2769692 = score(doc=4786,freq=1.0), product of:
              0.47645882 = queryWeight, product of:
                3.685713 = boost
                7.4407387 = idf(docFreq=68, maxDocs=43254)
                0.017373519 = queryNorm
              0.5813077 = fieldWeight in 4786, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.4407387 = idf(docFreq=68, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
          0.28987214 = weight(abstract_txt:evaluation in 4786) [ClassicSimilarity], result of:
            0.28987214 = score(doc=4786,freq=7.0), product of:
              0.3123203 = queryWeight, product of:
                4.0035496 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.017373519 = queryNorm
              0.92812455 = fieldWeight in 4786, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.078125 = fieldNorm(doc=4786)
        0.28 = coord(7/25)
    
  3. Névéol, A.; Deserno, T.M.; Darmoni, S.J.; Güld, M.O.; Aronson, A.R.: Natural language processing versus content-based image analysis for medical document retrieval (2009) 0.19
    0.19448648 = sum of:
      0.19448648 = product of:
        0.5402402 = sum of:
          0.012292898 = weight(abstract_txt:system in 4703) [ClassicSimilarity], result of:
            0.012292898 = score(doc=4703,freq=1.0), product of:
              0.05845626 = queryWeight, product of:
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.017373519 = queryNorm
              0.21029225 = fieldWeight in 4703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.027830822 = weight(abstract_txt:test in 4703) [ClassicSimilarity], result of:
            0.027830822 = score(doc=4703,freq=1.0), product of:
              0.08804711 = queryWeight, product of:
                1.0020661 = boost
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.017373519 = queryNorm
              0.31609014 = fieldWeight in 4703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.012836706 = weight(abstract_txt:systems in 4703) [ClassicSimilarity], result of:
            0.012836706 = score(doc=4703,freq=1.0), product of:
              0.060167767 = queryWeight, product of:
                1.0145335 = boost
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.017373519 = queryNorm
              0.21334855 = fieldWeight in 4703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.045854617 = weight(abstract_txt:indexing in 4703) [ClassicSimilarity], result of:
            0.045854617 = score(doc=4703,freq=3.0), product of:
              0.0974862 = queryWeight, product of:
                1.2913859 = boost
                4.345095 = idf(docFreq=1524, maxDocs=43254)
                0.017373519 = queryNorm
              0.47037035 = fieldWeight in 4703, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.345095 = idf(docFreq=1524, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.15026337 = weight(abstract_txt:image in 4703) [ClassicSimilarity], result of:
            0.15026337 = score(doc=4703,freq=9.0), product of:
              0.14912464 = queryWeight, product of:
                1.5971994 = boost
                5.374059 = idf(docFreq=544, maxDocs=43254)
                0.017373519 = queryNorm
              1.0076361 = fieldWeight in 4703, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                5.374059 = idf(docFreq=544, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.031100526 = weight(abstract_txt:methods in 4703) [ClassicSimilarity], result of:
            0.031100526 = score(doc=4703,freq=1.0), product of:
              0.11945919 = queryWeight, product of:
                1.6506828 = boost
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.017373519 = queryNorm
              0.26034436 = fieldWeight in 4703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.09694387 = weight(abstract_txt:images in 4703) [ClassicSimilarity], result of:
            0.09694387 = score(doc=4703,freq=2.0), product of:
              0.20232272 = queryWeight, product of:
                2.1482072 = boost
                5.421016 = idf(docFreq=519, maxDocs=43254)
                0.017373519 = queryNorm
              0.47915465 = fieldWeight in 4703, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.421016 = idf(docFreq=519, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.07546834 = weight(abstract_txt:automatic in 4703) [ClassicSimilarity], result of:
            0.07546834 = score(doc=4703,freq=1.0), product of:
              0.23237404 = queryWeight, product of:
                2.573965 = boost
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.017373519 = queryNorm
              0.32477096 = fieldWeight in 4703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
          0.08764909 = weight(abstract_txt:evaluation in 4703) [ClassicSimilarity], result of:
            0.08764909 = score(doc=4703,freq=1.0), product of:
              0.3123203 = queryWeight, product of:
                4.0035496 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.017373519 = queryNorm
              0.2806385 = fieldWeight in 4703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=4703)
        0.36 = coord(9/25)
    
  4. Rasmussen, E.M.: Indexing images (1997) 0.19
    0.18684353 = sum of:
      0.18684353 = product of:
        0.6672983 = sum of:
          0.032091767 = weight(abstract_txt:systems in 4216) [ClassicSimilarity], result of:
            0.032091767 = score(doc=4216,freq=4.0), product of:
              0.060167767 = queryWeight, product of:
                1.0145335 = boost
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.017373519 = queryNorm
              0.5333714 = fieldWeight in 4216, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
          0.081060275 = weight(abstract_txt:indexing in 4216) [ClassicSimilarity], result of:
            0.081060275 = score(doc=4216,freq=6.0), product of:
              0.0974862 = queryWeight, product of:
                1.2913859 = boost
                4.345095 = idf(docFreq=1524, maxDocs=43254)
                0.017373519 = queryNorm
              0.8315052 = fieldWeight in 4216, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                4.345095 = idf(docFreq=1524, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
          0.13999963 = weight(abstract_txt:image in 4216) [ClassicSimilarity], result of:
            0.13999963 = score(doc=4216,freq=5.0), product of:
              0.14912464 = queryWeight, product of:
                1.5971994 = boost
                5.374059 = idf(docFreq=544, maxDocs=43254)
                0.017373519 = queryNorm
              0.9388095 = fieldWeight in 4216, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.374059 = idf(docFreq=544, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
          0.038875658 = weight(abstract_txt:methods in 4216) [ClassicSimilarity], result of:
            0.038875658 = score(doc=4216,freq=1.0), product of:
              0.11945919 = queryWeight, product of:
                1.6506828 = boost
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.017373519 = queryNorm
              0.32543045 = fieldWeight in 4216, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
          0.17137419 = weight(abstract_txt:images in 4216) [ClassicSimilarity], result of:
            0.17137419 = score(doc=4216,freq=4.0), product of:
              0.20232272 = queryWeight, product of:
                2.1482072 = boost
                5.421016 = idf(docFreq=519, maxDocs=43254)
                0.017373519 = queryNorm
              0.8470338 = fieldWeight in 4216, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.421016 = idf(docFreq=519, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
          0.09433542 = weight(abstract_txt:automatic in 4216) [ClassicSimilarity], result of:
            0.09433542 = score(doc=4216,freq=1.0), product of:
              0.23237404 = queryWeight, product of:
                2.573965 = boost
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.017373519 = queryNorm
              0.4059637 = fieldWeight in 4216, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
          0.10956136 = weight(abstract_txt:evaluation in 4216) [ClassicSimilarity], result of:
            0.10956136 = score(doc=4216,freq=1.0), product of:
              0.3123203 = queryWeight, product of:
                4.0035496 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.017373519 = queryNorm
              0.3507981 = fieldWeight in 4216, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.078125 = fieldNorm(doc=4216)
        0.28 = coord(7/25)
    
  5. Gil-Leiva, I.: SISA-automatic indexing system for scientific articles : experiments with location heuristics rules versus TF-IDF rules (2017) 0.17
    0.1735428 = sum of:
      0.1735428 = product of:
        0.54232126 = sum of:
          0.015366123 = weight(abstract_txt:system in 5087) [ClassicSimilarity], result of:
            0.015366123 = score(doc=5087,freq=1.0), product of:
              0.05845626 = queryWeight, product of:
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.017373519 = queryNorm
              0.2628653 = fieldWeight in 5087, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.016045883 = weight(abstract_txt:systems in 5087) [ClassicSimilarity], result of:
            0.016045883 = score(doc=5087,freq=1.0), product of:
              0.060167767 = queryWeight, product of:
                1.0145335 = boost
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.017373519 = queryNorm
              0.2666857 = fieldWeight in 5087, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4135768 = idf(docFreq=3870, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.017306186 = weight(abstract_txt:results in 5087) [ClassicSimilarity], result of:
            0.017306186 = score(doc=5087,freq=1.0), product of:
              0.06327844 = queryWeight, product of:
                1.0404288 = boost
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.017373519 = queryNorm
              0.27349263 = fieldWeight in 5087, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5007057 = idf(docFreq=3547, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.020134958 = weight(abstract_txt:some in 5087) [ClassicSimilarity], result of:
            0.020134958 = score(doc=5087,freq=1.0), product of:
              0.06999849 = queryWeight, product of:
                1.0942808 = boost
                3.6819005 = idf(docFreq=2959, maxDocs=43254)
                0.017373519 = queryNorm
              0.28764847 = fieldWeight in 5087, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6819005 = idf(docFreq=2959, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.05936597 = weight(abstract_txt:keywords in 5087) [ClassicSimilarity], result of:
            0.05936597 = score(doc=5087,freq=1.0), product of:
              0.12573305 = queryWeight, product of:
                1.197467 = boost
                6.043633 = idf(docFreq=278, maxDocs=43254)
                0.017373519 = queryNorm
              0.47215882 = fieldWeight in 5087, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.043633 = idf(docFreq=278, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.09360034 = weight(abstract_txt:indexing in 5087) [ClassicSimilarity], result of:
            0.09360034 = score(doc=5087,freq=8.0), product of:
              0.0974862 = queryWeight, product of:
                1.2913859 = boost
                4.345095 = idf(docFreq=1524, maxDocs=43254)
                0.017373519 = queryNorm
              0.9601394 = fieldWeight in 5087, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                4.345095 = idf(docFreq=1524, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.21094042 = weight(abstract_txt:automatic in 5087) [ClassicSimilarity], result of:
            0.21094042 = score(doc=5087,freq=5.0), product of:
              0.23237404 = queryWeight, product of:
                2.573965 = boost
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.017373519 = queryNorm
              0.9077624 = fieldWeight in 5087, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.1963353 = idf(docFreq=650, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
          0.10956136 = weight(abstract_txt:evaluation in 5087) [ClassicSimilarity], result of:
            0.10956136 = score(doc=5087,freq=1.0), product of:
              0.3123203 = queryWeight, product of:
                4.0035496 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.017373519 = queryNorm
              0.3507981 = fieldWeight in 5087, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.078125 = fieldNorm(doc=5087)
        0.32 = coord(8/25)