Document (#32912)

Author
Sakai, T.
Title
On the reliability of information retrieval metrics based on graded relevance
Source
Information processing and management. 43(2007) no.2, S.531-548
Year
2007
Abstract
This paper compares 14 information retrieval metrics based on graded relevance, together with 10 traditional metrics based on binary relevance, in terms of stability, sensitivity and resemblance of system rankings. More specifically, we compare these metrics using the Buckley/Voorhees stability method, the Voorhees/Buckley swap method and Kendall's rank correlation, with three data sets comprising test collections and submitted runs from NTCIR. Our experiments show that (Average) Normalised Discounted Cumulative Gain at document cut-off l are the best among the rank-based graded-relevance metrics, provided that l is large. On the other hand, if one requires a recall-based graded-relevance metric that is highly correlated with Average Precision, then Q-measure is the best choice. Moreover, these best graded-relevance metrics are at least as stable and sensitive as Average Precision, and are fairly robust to the choice of gain values.
Footnote
Beitrag in: Special issue on AIRS2005: Information Retrieval Research in Asia
Theme
Retrievalalgorithmen

Similar documents (content)

  1. Kekäläinen, J.: Binary and graded relevance in IR evaluations : comparison of the effects on ranking of IR systems (2005) 0.59
    0.58554834 = sum of:
      0.58554834 = product of:
        1.3307917 = sum of:
          0.106170975 = weight(abstract_txt:binary in 3037) [ClassicSimilarity], result of:
            0.106170975 = score(doc=3037,freq=4.0), product of:
              0.09229469 = queryWeight, product of:
                1.0036848 = boost
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.012490227 = queryNorm
              1.1503476 = fieldWeight in 3037, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.055636108 = weight(abstract_txt:fairly in 3037) [ClassicSimilarity], result of:
            0.055636108 = score(doc=3037,freq=1.0), product of:
              0.09522786 = queryWeight, product of:
                1.0195088 = boost
                7.4782968 = idf(docFreq=64, maxDocs=42306)
                0.012490227 = queryNorm
              0.5842419 = fieldWeight in 3037, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.4782968 = idf(docFreq=64, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.009103156 = weight(abstract_txt:with in 3037) [ClassicSimilarity], result of:
            0.009103156 = score(doc=3037,freq=2.0), product of:
              0.032610036 = queryWeight, product of:
                1.0333447 = boost
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.012490227 = queryNorm
              0.27915198 = fieldWeight in 3037, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.13777246 = weight(abstract_txt:discounted in 3037) [ClassicSimilarity], result of:
            0.13777246 = score(doc=3037,freq=2.0), product of:
              0.1383432 = queryWeight, product of:
                1.2288189 = boost
                9.013627 = idf(docFreq=13, maxDocs=42306)
                0.012490227 = queryNorm
              0.99587446 = fieldWeight in 3037, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                9.013627 = idf(docFreq=13, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.11671476 = weight(abstract_txt:normalised in 3037) [ClassicSimilarity], result of:
            0.11671476 = score(doc=3037,freq=1.0), product of:
              0.15605465 = queryWeight, product of:
                1.3051106 = boost
                9.573242 = idf(docFreq=7, maxDocs=42306)
                0.012490227 = queryNorm
              0.74790955 = fieldWeight in 3037, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.573242 = idf(docFreq=7, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.044575408 = weight(abstract_txt:precision in 3037) [ClassicSimilarity], result of:
            0.044575408 = score(doc=3037,freq=1.0), product of:
              0.103498265 = queryWeight, product of:
                1.5031091 = boost
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.012490227 = queryNorm
              0.4306875 = fieldWeight in 3037, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.070453644 = weight(abstract_txt:rank in 3037) [ClassicSimilarity], result of:
            0.070453644 = score(doc=3037,freq=1.0), product of:
              0.14043385 = queryWeight, product of:
                1.7508941 = boost
                6.4215755 = idf(docFreq=186, maxDocs=42306)
                0.012490227 = queryNorm
              0.5016856 = fieldWeight in 3037, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.4215755 = idf(docFreq=186, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.15142347 = weight(abstract_txt:gain in 3037) [ClassicSimilarity], result of:
            0.15142347 = score(doc=3037,freq=4.0), product of:
              0.14733694 = queryWeight, product of:
                1.7934109 = boost
                6.5775104 = idf(docFreq=159, maxDocs=42306)
                0.012490227 = queryNorm
              1.027736 = fieldWeight in 3037, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.5775104 = idf(docFreq=159, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.022107314 = weight(abstract_txt:based in 3037) [ClassicSimilarity], result of:
            0.022107314 = score(doc=3037,freq=1.0), product of:
              0.08801165 = queryWeight, product of:
                2.191614 = boost
                3.2151837 = idf(docFreq=4616, maxDocs=42306)
                0.012490227 = queryNorm
              0.25118622 = fieldWeight in 3037, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2151837 = idf(docFreq=4616, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.19255711 = weight(abstract_txt:relevance in 3037) [ClassicSimilarity], result of:
            0.19255711 = score(doc=3037,freq=4.0), product of:
              0.24941905 = queryWeight, product of:
                4.0415635 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.012490227 = queryNorm
              0.7720225 = fieldWeight in 3037, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
          0.42427742 = weight(abstract_txt:graded in 3037) [ClassicSimilarity], result of:
            0.42427742 = score(doc=3037,freq=1.0), product of:
              0.63088393 = queryWeight, product of:
                5.867711 = boost
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.012490227 = queryNorm
              0.67251265 = fieldWeight in 3037, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.078125 = fieldNorm(doc=3037)
        0.44 = coord(11/25)
    
  2. Kekäläinen, J.; Järvelin, K.: Using graded relevance assessments in IR evaluation (2002) 0.26
    0.2621761 = sum of:
      0.2621761 = product of:
        0.81930035 = sum of:
          0.04246839 = weight(abstract_txt:binary in 226) [ClassicSimilarity], result of:
            0.04246839 = score(doc=226,freq=1.0), product of:
              0.09229469 = queryWeight, product of:
                1.0036848 = boost
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.012490227 = queryNorm
              0.46013904 = fieldWeight in 226, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.0072825253 = weight(abstract_txt:with in 226) [ClassicSimilarity], result of:
            0.0072825253 = score(doc=226,freq=2.0), product of:
              0.032610036 = queryWeight, product of:
                1.0333447 = boost
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.012490227 = queryNorm
              0.22332159 = fieldWeight in 226, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.01984747 = weight(abstract_txt:method in 226) [ClassicSimilarity], result of:
            0.01984747 = score(doc=226,freq=1.0), product of:
              0.07002927 = queryWeight, product of:
                1.2364136 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.012490227 = queryNorm
              0.28341675 = fieldWeight in 226, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.07132065 = weight(abstract_txt:precision in 226) [ClassicSimilarity], result of:
            0.07132065 = score(doc=226,freq=4.0), product of:
              0.103498265 = queryWeight, product of:
                1.5031091 = boost
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.012490227 = queryNorm
              0.68909997 = fieldWeight in 226, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.04122027 = weight(abstract_txt:best in 226) [ClassicSimilarity], result of:
            0.04122027 = score(doc=226,freq=1.0), product of:
              0.13049099 = queryWeight, product of:
                2.0670922 = boost
                5.0541754 = idf(docFreq=733, maxDocs=42306)
                0.012490227 = queryNorm
              0.31588596 = fieldWeight in 226, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0541754 = idf(docFreq=733, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.09395586 = weight(abstract_txt:average in 226) [ClassicSimilarity], result of:
            0.09395586 = score(doc=226,freq=2.0), product of:
              0.17938213 = queryWeight, product of:
                2.4235902 = boost
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.012490227 = queryNorm
              0.5237749 = fieldWeight in 226, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.20378327 = weight(abstract_txt:relevance in 226) [ClassicSimilarity], result of:
            0.20378327 = score(doc=226,freq=7.0), product of:
              0.24941905 = queryWeight, product of:
                4.0415635 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.012490227 = queryNorm
              0.81703174 = fieldWeight in 226, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
          0.33942193 = weight(abstract_txt:graded in 226) [ClassicSimilarity], result of:
            0.33942193 = score(doc=226,freq=1.0), product of:
              0.63088393 = queryWeight, product of:
                5.867711 = boost
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.012490227 = queryNorm
              0.5380101 = fieldWeight in 226, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.0625 = fieldNorm(doc=226)
        0.32 = coord(8/25)
    
  3. Talvensaari, T.; Juhola, M.; Laurikkala, J.; Järvelin, K.: Corpus-based cross-language information retrieval in retrieval of highly relevant documents (2007) 0.19
    0.19420853 = sum of:
      0.19420853 = product of:
        0.6936019 = sum of:
          0.0072825253 = weight(abstract_txt:with in 2140) [ClassicSimilarity], result of:
            0.0072825253 = score(doc=2140,freq=2.0), product of:
              0.032610036 = queryWeight, product of:
                1.0333447 = boost
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.012490227 = queryNorm
              0.22332159 = fieldWeight in 2140, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
          0.050972067 = weight(abstract_txt:runs in 2140) [ClassicSimilarity], result of:
            0.050972067 = score(doc=2140,freq=1.0), product of:
              0.10423678 = queryWeight, product of:
                1.066644 = boost
                7.824043 = idf(docFreq=45, maxDocs=42306)
                0.012490227 = queryNorm
              0.48900267 = fieldWeight in 2140, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.824043 = idf(docFreq=45, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
          0.050431315 = weight(abstract_txt:precision in 2140) [ClassicSimilarity], result of:
            0.050431315 = score(doc=2140,freq=2.0), product of:
              0.103498265 = queryWeight, product of:
                1.5031091 = boost
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.012490227 = queryNorm
              0.48726726 = fieldWeight in 2140, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
          0.025011573 = weight(abstract_txt:based in 2140) [ClassicSimilarity], result of:
            0.025011573 = score(doc=2140,freq=2.0), product of:
              0.08801165 = queryWeight, product of:
                2.191614 = boost
                3.2151837 = idf(docFreq=4616, maxDocs=42306)
                0.012490227 = queryNorm
              0.28418478 = fieldWeight in 2140, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2151837 = idf(docFreq=4616, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
          0.06643683 = weight(abstract_txt:average in 2140) [ClassicSimilarity], result of:
            0.06643683 = score(doc=2140,freq=1.0), product of:
              0.17938213 = queryWeight, product of:
                2.4235902 = boost
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.012490227 = queryNorm
              0.3703648 = fieldWeight in 2140, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
          0.15404569 = weight(abstract_txt:relevance in 2140) [ClassicSimilarity], result of:
            0.15404569 = score(doc=2140,freq=4.0), product of:
              0.24941905 = queryWeight, product of:
                4.0415635 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.012490227 = queryNorm
              0.61761796 = fieldWeight in 2140, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
          0.33942193 = weight(abstract_txt:graded in 2140) [ClassicSimilarity], result of:
            0.33942193 = score(doc=2140,freq=1.0), product of:
              0.63088393 = queryWeight, product of:
                5.867711 = boost
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.012490227 = queryNorm
              0.5380101 = fieldWeight in 2140, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.0625 = fieldNorm(doc=2140)
        0.28 = coord(7/25)
    
  4. Sun, A.; Bhowmick, S.S.; Nguyen, K.T.N.; Bai, G.: Tag-based social image retrieval : an empirical evaluation (2011) 0.19
    0.19074318 = sum of:
      0.19074318 = product of:
        0.52984214 = sum of:
          0.0063722096 = weight(abstract_txt:with in 1939) [ClassicSimilarity], result of:
            0.0063722096 = score(doc=1939,freq=2.0), product of:
              0.032610036 = queryWeight, product of:
                1.0333447 = boost
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.012490227 = queryNorm
              0.19540639 = fieldWeight in 1939, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.04132042 = weight(abstract_txt:cumulative in 1939) [ClassicSimilarity], result of:
            0.04132042 = score(doc=1939,freq=1.0), product of:
              0.099061295 = queryWeight, product of:
                1.0398268 = boost
                7.6273327 = idf(docFreq=55, maxDocs=42306)
                0.012490227 = queryNorm
              0.41711974 = fieldWeight in 1939, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.6273327 = idf(docFreq=55, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.06819389 = weight(abstract_txt:discounted in 1939) [ClassicSimilarity], result of:
            0.06819389 = score(doc=1939,freq=1.0), product of:
              0.1383432 = queryWeight, product of:
                1.2288189 = boost
                9.013627 = idf(docFreq=13, maxDocs=42306)
                0.012490227 = queryNorm
              0.49293274 = fieldWeight in 1939, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.013627 = idf(docFreq=13, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.0441274 = weight(abstract_txt:precision in 1939) [ClassicSimilarity], result of:
            0.0441274 = score(doc=1939,freq=2.0), product of:
              0.103498265 = queryWeight, product of:
                1.5031091 = boost
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.012490227 = queryNorm
              0.42635885 = fieldWeight in 1939, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.05299822 = weight(abstract_txt:gain in 1939) [ClassicSimilarity], result of:
            0.05299822 = score(doc=1939,freq=1.0), product of:
              0.14733694 = queryWeight, product of:
                1.7934109 = boost
                6.5775104 = idf(docFreq=159, maxDocs=42306)
                0.012490227 = queryNorm
              0.3597076 = fieldWeight in 1939, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.5775104 = idf(docFreq=159, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.021885125 = weight(abstract_txt:based in 1939) [ClassicSimilarity], result of:
            0.021885125 = score(doc=1939,freq=2.0), product of:
              0.08801165 = queryWeight, product of:
                2.191614 = boost
                3.2151837 = idf(docFreq=4616, maxDocs=42306)
                0.012490227 = queryNorm
              0.24866168 = fieldWeight in 1939, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.2151837 = idf(docFreq=4616, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.058132224 = weight(abstract_txt:average in 1939) [ClassicSimilarity], result of:
            0.058132224 = score(doc=1939,freq=1.0), product of:
              0.17938213 = queryWeight, product of:
                2.4235902 = boost
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.012490227 = queryNorm
              0.3240692 = fieldWeight in 1939, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.06739499 = weight(abstract_txt:relevance in 1939) [ClassicSimilarity], result of:
            0.06739499 = score(doc=1939,freq=1.0), product of:
              0.24941905 = queryWeight, product of:
                4.0415635 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.012490227 = queryNorm
              0.27020785 = fieldWeight in 1939, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
          0.16941768 = weight(abstract_txt:metrics in 1939) [ClassicSimilarity], result of:
            0.16941768 = score(doc=1939,freq=1.0), product of:
              0.4611233 = queryWeight, product of:
                5.4953218 = boost
                6.71821 = idf(docFreq=138, maxDocs=42306)
                0.012490227 = queryNorm
              0.36740214 = fieldWeight in 1939, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.71821 = idf(docFreq=138, maxDocs=42306)
                0.0546875 = fieldNorm(doc=1939)
        0.36 = coord(9/25)
    
  5. Bilal, D.: Ranking, relevance judgment, and precision of information retrieval on children's queries : evaluation of Google, Yahoo!, Bing, Yahoo! Kids, and ask Kids (2012) 0.19
    0.1856791 = sum of:
      0.1856791 = product of:
        0.66313964 = sum of:
          0.03801969 = weight(abstract_txt:metric in 2394) [ClassicSimilarity], result of:
            0.03801969 = score(doc=2394,freq=1.0), product of:
              0.09371301 = queryWeight, product of:
                1.0113673 = boost
                7.4185777 = idf(docFreq=68, maxDocs=42306)
                0.012490227 = queryNorm
              0.40570346 = fieldWeight in 2394, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.4185777 = idf(docFreq=68, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
          0.007804331 = weight(abstract_txt:with in 2394) [ClassicSimilarity], result of:
            0.007804331 = score(doc=2394,freq=3.0), product of:
              0.032610036 = queryWeight, product of:
                1.0333447 = boost
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.012490227 = queryNorm
              0.23932298 = fieldWeight in 2394, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.5265954 = idf(docFreq=9191, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
          0.0882548 = weight(abstract_txt:precision in 2394) [ClassicSimilarity], result of:
            0.0882548 = score(doc=2394,freq=8.0), product of:
              0.103498265 = queryWeight, product of:
                1.5031091 = boost
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.012490227 = queryNorm
              0.8527177 = fieldWeight in 2394, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                5.5127997 = idf(docFreq=463, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
          0.036067735 = weight(abstract_txt:best in 2394) [ClassicSimilarity], result of:
            0.036067735 = score(doc=2394,freq=1.0), product of:
              0.13049099 = queryWeight, product of:
                2.0670922 = boost
                5.0541754 = idf(docFreq=733, maxDocs=42306)
                0.012490227 = queryNorm
              0.2764002 = fieldWeight in 2394, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0541754 = idf(docFreq=733, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
          0.100687966 = weight(abstract_txt:average in 2394) [ClassicSimilarity], result of:
            0.100687966 = score(doc=2394,freq=3.0), product of:
              0.17938213 = queryWeight, product of:
                2.4235902 = boost
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.012490227 = queryNorm
              0.56130433 = fieldWeight in 2394, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.9258366 = idf(docFreq=306, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
          0.095310904 = weight(abstract_txt:relevance in 2394) [ClassicSimilarity], result of:
            0.095310904 = score(doc=2394,freq=2.0), product of:
              0.24941905 = queryWeight, product of:
                4.0415635 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.012490227 = queryNorm
              0.3821316 = fieldWeight in 2394, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
          0.2969942 = weight(abstract_txt:graded in 2394) [ClassicSimilarity], result of:
            0.2969942 = score(doc=2394,freq=1.0), product of:
              0.63088393 = queryWeight, product of:
                5.867711 = boost
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.012490227 = queryNorm
              0.47075886 = fieldWeight in 2394, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.608162 = idf(docFreq=20, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2394)
        0.28 = coord(7/25)