Document (#42055)

Author
Rodrigo, A.
Peñas, A.
Miyao, Y.
Kando, N.
Title
Do systems pass university entrance exams?
Source
Information processing and management. 54(2018) no.4, S.564-575
Year
2018
Abstract
Reading Comprehension tests are commonly used to assess the degree to which people comprehend what they read. This is why we work with the hypothesis that it is reasonable to use these tests to assess the degree to which a machine "comprehends" what it is reading. In this work, we evaluate Question Answering systems using Reading Comprehension tests from exams to enter University. This article analyses the datasets generated, the kind of inferences required, the methodology followed in three evaluation campaigns, the approaches presented by participants and current results. Besides, we study the evolution of systems and the main lessons learned in this evaluation process. We also show how current technologies are unable to pass university-entrance exams. This is because these tests require a deep understanding of texts, as well as detecting the similar meaning of phrases with different words. Future directions focused on these ideas seem more promising than including a massive amount of data for training systems, what has allowed systems to obtain outstanding results in Reading Comprehension tests with more straightforward questions. We think this study helps to increase the knowledge about how to develop better Question Answering systems.
Content
Vgl.: https://doi.org/10.1016/j.ipm.2018.03.002.
Theme
Information
Field
Informatik

Similar documents (author)

  1. Kando, N.: Information concepts reexamined (1994) 6.09 
    6.094361 = sum of:
  6.094361 = weight(author_txt:kando in 2126) [ClassicSimilarity], result of:
    6.094361 = fieldWeight in 2126, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.7509775 = idf(docFreq=6, maxDocs=44218)
      0.625 = fieldNorm(doc=2126)
  2. Hu, X.; Kando, N.: Task complexity and difficulty in music information retrieval (2017) 4.88 
    4.8754888 = sum of:
  4.8754888 = weight(author_txt:kando in 3690) [ClassicSimilarity], result of:
    4.8754888 = fieldWeight in 3690, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.7509775 = idf(docFreq=6, maxDocs=44218)
      0.5 = fieldNorm(doc=3690)
  3. Fujii, A.; Iwayama, M.; Kando, N.: Introduction to the special issue on patent processing (2007) 3.66 
    3.6566167 = sum of:
  3.6566167 = weight(author_txt:kando in 929) [ClassicSimilarity], result of:
    3.6566167 = fieldWeight in 929, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.7509775 = idf(docFreq=6, maxDocs=44218)
      0.375 = fieldNorm(doc=929)
  4. Gey, F.C.; Kando, N.; Peters, C.: Cross-Language Information Retrieval : the way ahead (2005) 3.66 
    3.6566167 = sum of:
  3.6566167 = weight(author_txt:kando in 1018) [ClassicSimilarity], result of:
    3.6566167 = fieldWeight in 1018, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.7509775 = idf(docFreq=6, maxDocs=44218)
      0.375 = fieldNorm(doc=1018)
  5. Kuriyama, K.; Kando, N.; Nozue, T.; Eguchi, K.: Pooling for a large-scale test collection : an analysis of the search results from the First NTCIR Workshop (2002) 3.05 
    3.0471804 = sum of:
  3.0471804 = weight(author_txt:kando in 3830) [ClassicSimilarity], result of:
    3.0471804 = fieldWeight in 3830, product of:
      1.0 = tf(freq=1.0), with freq of:
        1.0 = termFreq=1.0
      9.7509775 = idf(docFreq=6, maxDocs=44218)
      0.3125 = fieldNorm(doc=3830)

Similar documents (content)

  1. Wacholder, N.; Kelly, D.; Kantor, P.; Rittman, R.; Sun, Y.; Bai, B.; Small, S.; Yamrom, B.; Strzalkowski, T.: ¬A model for quantitative evaluation of an end-to-end question-answering system (2007) 0.16 
    0.15570016 = sum of:
  0.15570016 = product of:
    0.48656303 = sum of:
      0.008076284 = weight(abstract_txt:with in 435) [ClassicSimilarity], result of:
        0.008076284 = score(doc=435,freq=1.0), product of:
          0.041354984 = queryWeight, product of:
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.016543757 = queryNorm
          0.19529167 = fieldWeight in 435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.018830787 = weight(abstract_txt:work in 435) [ClassicSimilarity], result of:
        0.018830787 = score(doc=435,freq=1.0), product of:
          0.06352364 = queryWeight, product of:
            1.0119479 = boost
            3.7943997 = idf(docFreq=2703, maxDocs=44218)
            0.016543757 = queryNorm
          0.29643747 = fieldWeight in 435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.7943997 = idf(docFreq=2703, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.06223963 = weight(abstract_txt:evaluation in 435) [ClassicSimilarity], result of:
        0.06223963 = score(doc=435,freq=4.0), product of:
          0.08879347 = queryWeight, product of:
            1.1964126 = boost
            4.4860687 = idf(docFreq=1353, maxDocs=44218)
            0.016543757 = queryNorm
          0.70094824 = fieldWeight in 435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.4860687 = idf(docFreq=1353, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.06884742 = weight(abstract_txt:question in 435) [ClassicSimilarity], result of:
        0.06884742 = score(doc=435,freq=2.0), product of:
          0.119657 = queryWeight, product of:
            1.3888633 = boost
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.016543757 = queryNorm
          0.57537305 = fieldWeight in 435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.072517246 = weight(abstract_txt:assess in 435) [ClassicSimilarity], result of:
        0.072517246 = score(doc=435,freq=1.0), product of:
          0.15606922 = queryWeight, product of:
            1.5861676 = boost
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.016543757 = queryNorm
          0.46464798 = fieldWeight in 435, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.14485012 = weight(abstract_txt:answering in 435) [ClassicSimilarity], result of:
        0.14485012 = score(doc=435,freq=2.0), product of:
          0.1964682 = queryWeight, product of:
            1.7796582 = boost
            6.6730065 = idf(docFreq=151, maxDocs=44218)
            0.016543757 = queryNorm
          0.73727006 = fieldWeight in 435, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            6.6730065 = idf(docFreq=151, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.029058415 = weight(abstract_txt:this in 435) [ClassicSimilarity], result of:
        0.029058415 = score(doc=435,freq=4.0), product of:
          0.07707108 = queryWeight, product of:
            1.93062 = boost
            2.4130175 = idf(docFreq=10762, maxDocs=44218)
            0.016543757 = queryNorm
          0.37703398 = fieldWeight in 435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            2.4130175 = idf(docFreq=10762, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
      0.08214309 = weight(abstract_txt:systems in 435) [ClassicSimilarity], result of:
        0.08214309 = score(doc=435,freq=4.0), product of:
          0.15408398 = queryWeight, product of:
            2.7297935 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.016543757 = queryNorm
          0.53310597 = fieldWeight in 435, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.078125 = fieldNorm(doc=435)
    0.32 = coord(8/25)
  2. Wong, W.; Thangarajah, J.T.; Padgham, L.: Contextual question answering for the health domain (2012) 0.14 
    0.14227805 = sum of:
  0.14227805 = product of:
    0.50813586 = sum of:
      0.011190827 = weight(abstract_txt:with in 499) [ClassicSimilarity], result of:
        0.011190827 = score(doc=499,freq=3.0), product of:
          0.041354984 = queryWeight, product of:
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.016543757 = queryNorm
          0.27060407 = fieldWeight in 499, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
      0.07289462 = weight(abstract_txt:unable in 499) [ClassicSimilarity], result of:
        0.07289462 = score(doc=499,freq=1.0), product of:
          0.14423922 = queryWeight, product of:
            1.0782442 = boost
            8.085969 = idf(docFreq=36, maxDocs=44218)
            0.016543757 = queryNorm
          0.50537306 = fieldWeight in 499, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.085969 = idf(docFreq=36, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
      0.08931073 = weight(abstract_txt:comprehend in 499) [ClassicSimilarity], result of:
        0.08931073 = score(doc=499,freq=1.0), product of:
          0.16515389 = queryWeight, product of:
            1.1537716 = boost
            8.652365 = idf(docFreq=20, maxDocs=44218)
            0.016543757 = queryNorm
          0.5407728 = fieldWeight in 499, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            8.652365 = idf(docFreq=20, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
      0.07789195 = weight(abstract_txt:question in 499) [ClassicSimilarity], result of:
        0.07789195 = score(doc=499,freq=4.0), product of:
          0.119657 = queryWeight, product of:
            1.3888633 = boost
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.016543757 = queryNorm
          0.65096027 = fieldWeight in 499, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
      0.058013797 = weight(abstract_txt:assess in 499) [ClassicSimilarity], result of:
        0.058013797 = score(doc=499,freq=1.0), product of:
          0.15606922 = queryWeight, product of:
            1.5861676 = boost
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.016543757 = queryNorm
          0.37171838 = fieldWeight in 499, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
      0.14192355 = weight(abstract_txt:answering in 499) [ClassicSimilarity], result of:
        0.14192355 = score(doc=499,freq=3.0), product of:
          0.1964682 = queryWeight, product of:
            1.7796582 = boost
            6.6730065 = idf(docFreq=151, maxDocs=44218)
            0.016543757 = queryNorm
          0.72237414 = fieldWeight in 499, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.6730065 = idf(docFreq=151, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
      0.056910403 = weight(abstract_txt:systems in 499) [ClassicSimilarity], result of:
        0.056910403 = score(doc=499,freq=3.0), product of:
          0.15408398 = queryWeight, product of:
            2.7297935 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.016543757 = queryNorm
          0.36934665 = fieldWeight in 499, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.0625 = fieldNorm(doc=499)
    0.28 = coord(7/25)
  3. Parapar, J.; Losada, D.E.; Presedo-Quindimil, M.A.; Barreiro, A.: Using score distributions to compare statistical significance tests for information retrieval evaluation (2020) 0.14 
    0.14182918 = sum of:
  0.14182918 = product of:
    0.5909549 = sum of:
      0.02189571 = weight(abstract_txt:current in 5506) [ClassicSimilarity], result of:
        0.02189571 = score(doc=5506,freq=1.0), product of:
          0.081508406 = queryWeight, product of:
            1.1462824 = boost
            4.298101 = idf(docFreq=1633, maxDocs=44218)
            0.016543757 = queryNorm
          0.2686313 = fieldWeight in 5506, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.298101 = idf(docFreq=1633, maxDocs=44218)
            0.0625 = fieldNorm(doc=5506)
      0.03520805 = weight(abstract_txt:evaluation in 5506) [ClassicSimilarity], result of:
        0.03520805 = score(doc=5506,freq=2.0), product of:
          0.08879347 = queryWeight, product of:
            1.1964126 = boost
            4.4860687 = idf(docFreq=1353, maxDocs=44218)
            0.016543757 = queryNorm
          0.3965162 = fieldWeight in 5506, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.4860687 = idf(docFreq=1353, maxDocs=44218)
            0.0625 = fieldNorm(doc=5506)
      0.058013797 = weight(abstract_txt:assess in 5506) [ClassicSimilarity], result of:
        0.058013797 = score(doc=5506,freq=1.0), product of:
          0.15606922 = queryWeight, product of:
            1.5861676 = boost
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.016543757 = queryNorm
          0.37171838 = fieldWeight in 5506, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.0625 = fieldNorm(doc=5506)
      0.02013226 = weight(abstract_txt:this in 5506) [ClassicSimilarity], result of:
        0.02013226 = score(doc=5506,freq=3.0), product of:
          0.07707108 = queryWeight, product of:
            1.93062 = boost
            2.4130175 = idf(docFreq=10762, maxDocs=44218)
            0.016543757 = queryNorm
          0.2612168 = fieldWeight in 5506, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            2.4130175 = idf(docFreq=10762, maxDocs=44218)
            0.0625 = fieldNorm(doc=5506)
      0.04646715 = weight(abstract_txt:systems in 5506) [ClassicSimilarity], result of:
        0.04646715 = score(doc=5506,freq=2.0), product of:
          0.15408398 = queryWeight, product of:
            2.7297935 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.016543757 = queryNorm
          0.3015703 = fieldWeight in 5506, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.0625 = fieldNorm(doc=5506)
      0.40923792 = weight(abstract_txt:tests in 5506) [ClassicSimilarity], result of:
        0.40923792 = score(doc=5506,freq=7.0), product of:
          0.40728113 = queryWeight, product of:
            4.051423 = boost
            6.0764866 = idf(docFreq=275, maxDocs=44218)
            0.016543757 = queryNorm
          1.0048045 = fieldWeight in 5506, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            6.0764866 = idf(docFreq=275, maxDocs=44218)
            0.0625 = fieldNorm(doc=5506)
    0.24 = coord(6/25)
  4. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amode, D.; Sutskever, I.: Language models are unsupervised multitask learners 0.14 
    0.13637994 = sum of:
  0.13637994 = product of:
    0.48707125 = sum of:
      0.006461027 = weight(abstract_txt:with in 871) [ClassicSimilarity], result of:
        0.006461027 = score(doc=871,freq=1.0), product of:
          0.041354984 = queryWeight, product of:
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.016543757 = queryNorm
          0.15623334 = fieldWeight in 871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
      0.02321608 = weight(abstract_txt:these in 871) [ClassicSimilarity], result of:
        0.02321608 = score(doc=871,freq=3.0), product of:
          0.06726859 = queryWeight, product of:
            1.2753878 = boost
            3.1881294 = idf(docFreq=4957, maxDocs=44218)
            0.016543757 = queryNorm
          0.34512514 = fieldWeight in 871, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.1881294 = idf(docFreq=4957, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
      0.038945977 = weight(abstract_txt:question in 871) [ClassicSimilarity], result of:
        0.038945977 = score(doc=871,freq=1.0), product of:
          0.119657 = queryWeight, product of:
            1.3888633 = boost
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.016543757 = queryNorm
          0.32548013 = fieldWeight in 871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
      0.0819396 = weight(abstract_txt:answering in 871) [ClassicSimilarity], result of:
        0.0819396 = score(doc=871,freq=1.0), product of:
          0.1964682 = queryWeight, product of:
            1.7796582 = boost
            6.6730065 = idf(docFreq=151, maxDocs=44218)
            0.016543757 = queryNorm
          0.4170629 = fieldWeight in 871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.6730065 = idf(docFreq=151, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
      0.04646715 = weight(abstract_txt:systems in 871) [ClassicSimilarity], result of:
        0.04646715 = score(doc=871,freq=2.0), product of:
          0.15408398 = queryWeight, product of:
            2.7297935 = boost
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.016543757 = queryNorm
          0.3015703 = fieldWeight in 871, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.4118783 = idf(docFreq=3963, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
      0.17092596 = weight(abstract_txt:comprehension in 871) [ClassicSimilarity], result of:
        0.17092596 = score(doc=871,freq=1.0), product of:
          0.36716858 = queryWeight, product of:
            2.9796746 = boost
            7.448392 = idf(docFreq=69, maxDocs=44218)
            0.016543757 = queryNorm
          0.4655245 = fieldWeight in 871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.448392 = idf(docFreq=69, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
      0.11911545 = weight(abstract_txt:reading in 871) [ClassicSimilarity], result of:
        0.11911545 = score(doc=871,freq=1.0), product of:
          0.31765214 = queryWeight, product of:
            3.200231 = boost
            5.9997935 = idf(docFreq=297, maxDocs=44218)
            0.016543757 = queryNorm
          0.3749871 = fieldWeight in 871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.9997935 = idf(docFreq=297, maxDocs=44218)
            0.0625 = fieldNorm(doc=871)
    0.28 = coord(7/25)
  5. Chen, A.T.: Information use and illness representations : understanding their connection in illness coping (2015) 0.13 
    0.13487491 = sum of:
  0.13487491 = product of:
    0.48169613 = sum of:
      0.009137273 = weight(abstract_txt:with in 1640) [ClassicSimilarity], result of:
        0.009137273 = score(doc=1640,freq=2.0), product of:
          0.041354984 = queryWeight, product of:
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.016543757 = queryNorm
          0.22094731 = fieldWeight in 1640, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
      0.013403811 = weight(abstract_txt:these in 1640) [ClassicSimilarity], result of:
        0.013403811 = score(doc=1640,freq=1.0), product of:
          0.06726859 = queryWeight, product of:
            1.2753878 = boost
            3.1881294 = idf(docFreq=4957, maxDocs=44218)
            0.016543757 = queryNorm
          0.19925809 = fieldWeight in 1640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.1881294 = idf(docFreq=4957, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
      0.038945977 = weight(abstract_txt:question in 1640) [ClassicSimilarity], result of:
        0.038945977 = score(doc=1640,freq=1.0), product of:
          0.119657 = queryWeight, product of:
            1.3888633 = boost
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.016543757 = queryNorm
          0.32548013 = fieldWeight in 1640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.207682 = idf(docFreq=657, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
      0.033227015 = weight(abstract_txt:what in 1640) [ClassicSimilarity], result of:
        0.033227015 = score(doc=1640,freq=1.0), product of:
          0.12321238 = queryWeight, product of:
            1.7260892 = boost
            4.314763 = idf(docFreq=1606, maxDocs=44218)
            0.016543757 = queryNorm
          0.2696727 = fieldWeight in 1640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.314763 = idf(docFreq=1606, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
      0.025990639 = weight(abstract_txt:this in 1640) [ClassicSimilarity], result of:
        0.025990639 = score(doc=1640,freq=5.0), product of:
          0.07707108 = queryWeight, product of:
            1.93062 = boost
            2.4130175 = idf(docFreq=10762, maxDocs=44218)
            0.016543757 = queryNorm
          0.33722946 = fieldWeight in 1640, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            2.4130175 = idf(docFreq=10762, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
      0.20631401 = weight(abstract_txt:reading in 1640) [ClassicSimilarity], result of:
        0.20631401 = score(doc=1640,freq=3.0), product of:
          0.31765214 = queryWeight, product of:
            3.200231 = boost
            5.9997935 = idf(docFreq=297, maxDocs=44218)
            0.016543757 = queryNorm
          0.6494967 = fieldWeight in 1640, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.9997935 = idf(docFreq=297, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
      0.15467739 = weight(abstract_txt:tests in 1640) [ClassicSimilarity], result of:
        0.15467739 = score(doc=1640,freq=1.0), product of:
          0.40728113 = queryWeight, product of:
            4.051423 = boost
            6.0764866 = idf(docFreq=275, maxDocs=44218)
            0.016543757 = queryNorm
          0.3797804 = fieldWeight in 1640, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.0764866 = idf(docFreq=275, maxDocs=44218)
            0.0625 = fieldNorm(doc=1640)
    0.28 = coord(7/25)