Document (#42508)

Author
Parapar, J.
Losada, D.E.
Presedo-Quindimil, M.A.
Barreiro, A.
Title
Using score distributions to compare statistical significance tests for information retrieval evaluation
Source
Journal of the Association for Information Science and Technology. 71(2020) no.1, S.98-113
Year
2020
Abstract
Statistical significance tests can provide evidence that the observed difference in performance between 2 methods is not due to chance. In information retrieval (IR), some studies have examined the validity and suitability of such tests for comparing search systems. We argue here that current methods for assessing the reliability of statistical tests suffer from some methodological weaknesses, and we propose a novel way to study significance tests for retrieval evaluation. Using Score Distributions, we model the output of multiple search systems, produce simulated search results from such models, and compare them using various significance tests. A key strength of this approach is that we assess statistical tests under perfect knowledge about the truth or falseness of the null hypothesis. This new method for studying the power of significance tests in IR evaluation is formal and innovative. Following this type of analysis, we found that both the sign test and Wilcoxon signed test have more power than the permutation test and the t-test. The sign test and Wilcoxon signed test also have good behavior in terms of type I errors. The bootstrap test shows few type I errors, but it has less power than the other methods tested.
Content
Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24203.
Theme
Retrievalstudien

Similar documents (author)

  1. Losada, D.E.; Barreiro, A.: Emebedding term similarity and inverse document frequency into a logical model of information retrieval (2003) 4.86
    4.8644676 = sum of:
      4.8644676 = weight(author_txt:losada in 3423) [ClassicSimilarity], result of:
        4.8644676 = fieldWeight in 3423, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.728935 = idf(docFreq=6, maxDocs=43254)
          0.5 = fieldNorm(doc=3423)
    
  2. Fernández, R.T.; Losada, D.E.: Effective sentence retrieval based on query-independent evidence (2012) 4.86
    4.8644676 = sum of:
      4.8644676 = weight(author_txt:losada in 4193) [ClassicSimilarity], result of:
        4.8644676 = fieldWeight in 4193, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.728935 = idf(docFreq=6, maxDocs=43254)
          0.5 = fieldNorm(doc=4193)
    
  3. Sweeney, S.; Crestani, F.; Losada, D.E.: 'Show me more' : incremental length summarisation using novelty detection (2008) 3.65
    3.6483507 = sum of:
      3.6483507 = weight(author_txt:losada in 4055) [ClassicSimilarity], result of:
        3.6483507 = fieldWeight in 4055, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.728935 = idf(docFreq=6, maxDocs=43254)
          0.375 = fieldNorm(doc=4055)
    
  4. Losada, D.E.; Parapar, J.; Barreiro, A.: When to stop making relevance judgments? : a study of stopping methods for building information retrieval test collections (2019) 3.65
    3.6483507 = sum of:
      3.6483507 = weight(author_txt:losada in 675) [ClassicSimilarity], result of:
        3.6483507 = fieldWeight in 675, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.728935 = idf(docFreq=6, maxDocs=43254)
          0.375 = fieldNorm(doc=675)
    
  5. Losada, D.E.; Parapar, J.; Barreiro, A.: Multi-armed bandits for adjudicating documents in pooling-based evaluation of information retrieval systems (2017) 3.65
    3.6483507 = sum of:
      3.6483507 = weight(author_txt:losada in 99) [ClassicSimilarity], result of:
        3.6483507 = fieldWeight in 99, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.728935 = idf(docFreq=6, maxDocs=43254)
          0.375 = fieldNorm(doc=99)
    

Similar documents (content)

  1. Ravana, S.D.; Taheri, M.S.; Rajagopal, P.: Document-based approach to improve the accuracy of pairwise comparison in evaluating information retrieval systems (2015) 0.34
    0.34083033 = sum of:
      0.34083033 = product of:
        0.8520758 = sum of:
          0.0061095636 = weight(abstract_txt:that in 4052) [ClassicSimilarity], result of:
            0.0061095636 = score(doc=4052,freq=1.0), product of:
              0.040979438 = queryWeight, product of:
                1.2194293 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.014087866 = queryNorm
              0.14908852 = fieldWeight in 4052, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.01132621 = weight(abstract_txt:have in 4052) [ClassicSimilarity], result of:
            0.01132621 = score(doc=4052,freq=1.0), product of:
              0.056187022 = queryWeight, product of:
                1.2365801 = boost
                3.2252884 = idf(docFreq=4672, maxDocs=43254)
                0.014087866 = queryNorm
              0.20158052 = fieldWeight in 4052, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2252884 = idf(docFreq=4672, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.019945491 = weight(abstract_txt:retrieval in 4052) [ClassicSimilarity], result of:
            0.019945491 = score(doc=4052,freq=2.0), product of:
              0.06503285 = queryWeight, product of:
                1.3303645 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.014087866 = queryNorm
              0.3066987 = fieldWeight in 4052, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.024497582 = weight(abstract_txt:using in 4052) [ClassicSimilarity], result of:
            0.024497582 = score(doc=4052,freq=3.0), product of:
              0.06515604 = queryWeight, product of:
                1.3316239 = boost
                3.4731848 = idf(docFreq=3646, maxDocs=43254)
                0.014087866 = queryNorm
              0.37598327 = fieldWeight in 4052, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4731848 = idf(docFreq=3646, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.052934792 = weight(abstract_txt:evaluation in 4052) [ClassicSimilarity], result of:
            0.052934792 = score(doc=4052,freq=3.0), product of:
              0.108901374 = queryWeight, product of:
                1.7215551 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.014087866 = queryNorm
              0.4860801 = fieldWeight in 4052, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.0788554 = weight(abstract_txt:score in 4052) [ClassicSimilarity], result of:
            0.0788554 = score(doc=4052,freq=1.0), product of:
              0.17896585 = queryWeight, product of:
                1.801954 = boost
                7.0498724 = idf(docFreq=101, maxDocs=43254)
                0.014087866 = queryNorm
              0.44061702 = fieldWeight in 4052, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.0498724 = idf(docFreq=101, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.0767731 = weight(abstract_txt:statistical in 4052) [ClassicSimilarity], result of:
            0.0767731 = score(doc=4052,freq=1.0), product of:
              0.22149567 = queryWeight, product of:
                2.8350208 = boost
                5.545795 = idf(docFreq=458, maxDocs=43254)
                0.014087866 = queryNorm
              0.3466122 = fieldWeight in 4052, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.545795 = idf(docFreq=458, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.20391084 = weight(abstract_txt:significance in 4052) [ClassicSimilarity], result of:
            0.20391084 = score(doc=4052,freq=2.0), product of:
              0.36320022 = queryWeight, product of:
                4.0588374 = boost
                6.3518353 = idf(docFreq=204, maxDocs=43254)
                0.014087866 = queryNorm
              0.5614282 = fieldWeight in 4052, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.3518353 = idf(docFreq=204, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.17648552 = weight(abstract_txt:test in 4052) [ClassicSimilarity], result of:
            0.17648552 = score(doc=4052,freq=3.0), product of:
              0.32235733 = queryWeight, product of:
                4.5244045 = boost
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.014087866 = queryNorm
              0.54748416 = fieldWeight in 4052, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
          0.20123725 = weight(abstract_txt:tests in 4052) [ClassicSimilarity], result of:
            0.20123725 = score(doc=4052,freq=1.0), product of:
              0.5305278 = queryWeight, product of:
                6.2050123 = boost
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.014087866 = queryNorm
              0.3793152 = fieldWeight in 4052, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.0625 = fieldNorm(doc=4052)
        0.4 = coord(10/25)
    
  2. Drabenstott, K.M.; Weller, M.S.: ¬A comparative approach to system evaluation : delegating control of retrieval tests to an experimental online system (1996) 0.23
    0.22573434 = sum of:
      0.22573434 = product of:
        0.80619407 = sum of:
          0.009164345 = weight(abstract_txt:that in 505) [ClassicSimilarity], result of:
            0.009164345 = score(doc=505,freq=1.0), product of:
              0.040979438 = queryWeight, product of:
                1.2194293 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.014087866 = queryNorm
              0.22363278 = fieldWeight in 505, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
          0.036642205 = weight(abstract_txt:retrieval in 505) [ClassicSimilarity], result of:
            0.036642205 = score(doc=505,freq=3.0), product of:
              0.06503285 = queryWeight, product of:
                1.3303645 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.014087866 = queryNorm
              0.5634415 = fieldWeight in 505, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
          0.034906793 = weight(abstract_txt:search in 505) [ClassicSimilarity], result of:
            0.034906793 = score(doc=505,freq=2.0), product of:
              0.07207458 = queryWeight, product of:
                1.4005393 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.014087866 = queryNorm
              0.48431495 = fieldWeight in 505, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
          0.036599454 = weight(abstract_txt:methods in 505) [ClassicSimilarity], result of:
            0.036599454 = score(doc=505,freq=1.0), product of:
              0.09372063 = queryWeight, product of:
                1.5970622 = boost
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.014087866 = queryNorm
              0.39051652 = fieldWeight in 505, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
          0.04584288 = weight(abstract_txt:evaluation in 505) [ClassicSimilarity], result of:
            0.04584288 = score(doc=505,freq=1.0), product of:
              0.108901374 = queryWeight, product of:
                1.7215551 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.014087866 = queryNorm
              0.42095774 = fieldWeight in 505, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
          0.21614975 = weight(abstract_txt:test in 505) [ClassicSimilarity], result of:
            0.21614975 = score(doc=505,freq=2.0), product of:
              0.32235733 = queryWeight, product of:
                4.5244045 = boost
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.014087866 = queryNorm
              0.6705284 = fieldWeight in 505, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
          0.42688864 = weight(abstract_txt:tests in 505) [ClassicSimilarity], result of:
            0.42688864 = score(doc=505,freq=2.0), product of:
              0.5305278 = queryWeight, product of:
                6.2050123 = boost
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.014087866 = queryNorm
              0.804649 = fieldWeight in 505, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.09375 = fieldNorm(doc=505)
        0.28 = coord(7/25)
    
  3. Pirkola, A.; Järvelin, K.: Employing the resolution power of search keys (2001) 0.20
    0.19933696 = sum of:
      0.19933696 = product of:
        0.622928 = sum of:
          0.012960342 = weight(abstract_txt:that in 908) [ClassicSimilarity], result of:
            0.012960342 = score(doc=908,freq=2.0), product of:
              0.040979438 = queryWeight, product of:
                1.2194293 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.014087866 = queryNorm
              0.3162645 = fieldWeight in 908, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.016989315 = weight(abstract_txt:have in 908) [ClassicSimilarity], result of:
            0.016989315 = score(doc=908,freq=1.0), product of:
              0.056187022 = queryWeight, product of:
                1.2365801 = boost
                3.2252884 = idf(docFreq=4672, maxDocs=43254)
                0.014087866 = queryNorm
              0.3023708 = fieldWeight in 908, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2252884 = idf(docFreq=4672, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.021155387 = weight(abstract_txt:retrieval in 908) [ClassicSimilarity], result of:
            0.021155387 = score(doc=908,freq=1.0), product of:
              0.06503285 = queryWeight, product of:
                1.3303645 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.014087866 = queryNorm
              0.3253031 = fieldWeight in 908, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.02121553 = weight(abstract_txt:using in 908) [ClassicSimilarity], result of:
            0.02121553 = score(doc=908,freq=1.0), product of:
              0.06515604 = queryWeight, product of:
                1.3316239 = boost
                3.4731848 = idf(docFreq=3646, maxDocs=43254)
                0.014087866 = queryNorm
              0.32561108 = fieldWeight in 908, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4731848 = idf(docFreq=3646, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.042751916 = weight(abstract_txt:search in 908) [ClassicSimilarity], result of:
            0.042751916 = score(doc=908,freq=3.0), product of:
              0.07207458 = queryWeight, product of:
                1.4005393 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.014087866 = queryNorm
              0.59316224 = fieldWeight in 908, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.036599454 = weight(abstract_txt:methods in 908) [ClassicSimilarity], result of:
            0.036599454 = score(doc=908,freq=1.0), product of:
              0.09372063 = queryWeight, product of:
                1.5970622 = boost
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.014087866 = queryNorm
              0.39051652 = fieldWeight in 908, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.16940022 = weight(abstract_txt:power in 908) [ClassicSimilarity], result of:
            0.16940022 = score(doc=908,freq=3.0), product of:
              0.18047665 = queryWeight, product of:
                2.2162297 = boost
                5.780442 = idf(docFreq=362, maxDocs=43254)
                0.014087866 = queryNorm
              0.93862677 = fieldWeight in 908, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.780442 = idf(docFreq=362, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
          0.3018559 = weight(abstract_txt:tests in 908) [ClassicSimilarity], result of:
            0.3018559 = score(doc=908,freq=1.0), product of:
              0.5305278 = queryWeight, product of:
                6.2050123 = boost
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.014087866 = queryNorm
              0.5689728 = fieldWeight in 908, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.09375 = fieldNorm(doc=908)
        0.32 = coord(8/25)
    
  4. Salton, G.: ¬The state of retrieval system evaluation (1992) 0.18
    0.1841921 = sum of:
      0.1841921 = product of:
        0.76746714 = sum of:
          0.012219127 = weight(abstract_txt:that in 251) [ClassicSimilarity], result of:
            0.012219127 = score(doc=251,freq=1.0), product of:
              0.040979438 = queryWeight, product of:
                1.2194293 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.014087866 = queryNorm
              0.29817703 = fieldWeight in 251, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.125 = fieldNorm(doc=251)
          0.02265242 = weight(abstract_txt:have in 251) [ClassicSimilarity], result of:
            0.02265242 = score(doc=251,freq=1.0), product of:
              0.056187022 = queryWeight, product of:
                1.2365801 = boost
                3.2252884 = idf(docFreq=4672, maxDocs=43254)
                0.014087866 = queryNorm
              0.40316105 = fieldWeight in 251, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.2252884 = idf(docFreq=4672, maxDocs=43254)
                0.125 = fieldNorm(doc=251)
          0.039890982 = weight(abstract_txt:retrieval in 251) [ClassicSimilarity], result of:
            0.039890982 = score(doc=251,freq=2.0), product of:
              0.06503285 = queryWeight, product of:
                1.3303645 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.014087866 = queryNorm
              0.6133974 = fieldWeight in 251, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.125 = fieldNorm(doc=251)
          0.08644215 = weight(abstract_txt:evaluation in 251) [ClassicSimilarity], result of:
            0.08644215 = score(doc=251,freq=2.0), product of:
              0.108901374 = queryWeight, product of:
                1.7215551 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.014087866 = queryNorm
              0.7937655 = fieldWeight in 251, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.125 = fieldNorm(doc=251)
          0.20378794 = weight(abstract_txt:test in 251) [ClassicSimilarity], result of:
            0.20378794 = score(doc=251,freq=1.0), product of:
              0.32235733 = queryWeight, product of:
                4.5244045 = boost
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.014087866 = queryNorm
              0.6321803 = fieldWeight in 251, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.125 = fieldNorm(doc=251)
          0.4024745 = weight(abstract_txt:tests in 251) [ClassicSimilarity], result of:
            0.4024745 = score(doc=251,freq=1.0), product of:
              0.5305278 = queryWeight, product of:
                6.2050123 = boost
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.014087866 = queryNorm
              0.7586304 = fieldWeight in 251, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.125 = fieldNorm(doc=251)
        0.24 = coord(6/25)
    
  5. Kishida, K.: Term disambiguation techniques based on target document collection for cross-language information retrieval : an empirical comparison of performance between techniques (2007) 0.16
    0.16388744 = sum of:
      0.16388744 = product of:
        0.51214826 = sum of:
          0.010582074 = weight(abstract_txt:that in 2898) [ClassicSimilarity], result of:
            0.010582074 = score(doc=2898,freq=3.0), product of:
              0.040979438 = queryWeight, product of:
                1.2194293 = boost
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.014087866 = queryNorm
              0.25822887 = fieldWeight in 2898, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.3854163 = idf(docFreq=10822, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.014103591 = weight(abstract_txt:retrieval in 2898) [ClassicSimilarity], result of:
            0.014103591 = score(doc=2898,freq=1.0), product of:
              0.06503285 = queryWeight, product of:
                1.3303645 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.014087866 = queryNorm
              0.21686874 = fieldWeight in 2898, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.024497582 = weight(abstract_txt:using in 2898) [ClassicSimilarity], result of:
            0.024497582 = score(doc=2898,freq=3.0), product of:
              0.06515604 = queryWeight, product of:
                1.3316239 = boost
                3.4731848 = idf(docFreq=3646, maxDocs=43254)
                0.014087866 = queryNorm
              0.37598327 = fieldWeight in 2898, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4731848 = idf(docFreq=3646, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.016455222 = weight(abstract_txt:search in 2898) [ClassicSimilarity], result of:
            0.016455222 = score(doc=2898,freq=1.0), product of:
              0.07207458 = queryWeight, product of:
                1.4005393 = boost
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.014087866 = queryNorm
              0.22830826 = fieldWeight in 2898, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6529322 = idf(docFreq=3046, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.024399636 = weight(abstract_txt:methods in 2898) [ClassicSimilarity], result of:
            0.024399636 = score(doc=2898,freq=1.0), product of:
              0.09372063 = queryWeight, product of:
                1.5970622 = boost
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.014087866 = queryNorm
              0.26034436 = fieldWeight in 2898, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1655097 = idf(docFreq=1824, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.0767731 = weight(abstract_txt:statistical in 2898) [ClassicSimilarity], result of:
            0.0767731 = score(doc=2898,freq=1.0), product of:
              0.22149567 = queryWeight, product of:
                2.8350208 = boost
                5.545795 = idf(docFreq=458, maxDocs=43254)
                0.014087866 = queryNorm
              0.3466122 = fieldWeight in 2898, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.545795 = idf(docFreq=458, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.14409983 = weight(abstract_txt:test in 2898) [ClassicSimilarity], result of:
            0.14409983 = score(doc=2898,freq=2.0), product of:
              0.32235733 = queryWeight, product of:
                4.5244045 = boost
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.014087866 = queryNorm
              0.44701895 = fieldWeight in 2898, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.057442 = idf(docFreq=747, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
          0.20123725 = weight(abstract_txt:tests in 2898) [ClassicSimilarity], result of:
            0.20123725 = score(doc=2898,freq=1.0), product of:
              0.5305278 = queryWeight, product of:
                6.2050123 = boost
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.014087866 = queryNorm
              0.3793152 = fieldWeight in 2898, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.069043 = idf(docFreq=271, maxDocs=43254)
                0.0625 = fieldNorm(doc=2898)
        0.32 = coord(8/25)