Document (#36541)

Author

Chu, H.

Title

Factors affecting relevance judgment : a report from TREC Legal track

Source

Journal of documentation. 67(2011) no.2, S.264-278

Year

2011

Abstract

Purpose - This study intends to identify factors that affect relevance judgment of retrieved information as part of the 2007 TREC Legal track interactive task. Design/methodology/approach - Data were gathered and analyzed from the participants of the 2007 TREC Legal track interactive task using a questionnaire which includes not only a list of 80 relevance factors identified in prior research, but also a space for expressing their thoughts on relevance judgment in the process. Findings - This study finds that topicality remains a primary criterion, out of various options, for determining relevance, while specificity of the search request, task, or retrieved results also helps greatly in relevance judgment. Research limitations/implications - Relevance research should focus on the topicality and specificity of what is being evaluated as well as conducted in real environments. Practical implications - If multiple relevance factors are presented to assessors, the total number in a list should be below ten to take account of the limited processing capacity of human beings' short-term memory. Otherwise, the assessors might either completely ignore or inadequately consider some of the relevance factors when making judgment decisions. Originality/value - This study presents a method for reducing the artificiality of relevance research design, an apparent limitation in many related studies. Specifically, relevance judgment was made in this research as part of the 2007 TREC Legal track interactive task rather than a study devised for the sake of it. The assessors also served as searchers so that their searching experience would facilitate their subsequent relevance judgments.

Theme

Retrievalstudien

Object

TREC

Similar documents (content)

1. Ruthven, I.; Baillie, M.; Elsweiler, D.: ¬The relative effects of knowledge, interest and confidence in assessing relevance (2007) 0.52

   0.51602745 = sum of:
     0.51602745 = product of:
       0.9923605 = sum of:
         0.014101368 = weight(abstract_txt:should in 835) [ClassicSimilarity], result of:
           0.014101368 = score(doc=835,freq=1.0), product of:
             0.058168713 = queryWeight, product of:
               1.0729543 = boost
               4.432857 = idf(docFreq=1427, maxDocs=44218)
               0.012229945 = queryNorm
             0.24242187 = fieldWeight in 835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.432857 = idf(docFreq=1427, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.025239667 = weight(abstract_txt:implications in 835) [ClassicSimilarity], result of:
           0.025239667 = score(doc=835,freq=3.0), product of:
             0.059456218 = queryWeight, product of:
               1.0847636 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.012229945 = queryNorm
             0.42450845 = fieldWeight in 835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.015557747 = weight(abstract_txt:part in 835) [ClassicSimilarity], result of:
           0.015557747 = score(doc=835,freq=1.0), product of:
             0.062107842 = queryWeight, product of:
               1.108689 = boost
               4.580493 = idf(docFreq=1231, maxDocs=44218)
               0.012229945 = queryNorm
             0.2504957 = fieldWeight in 835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.580493 = idf(docFreq=1231, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.013265292 = weight(abstract_txt:their in 835) [ClassicSimilarity], result of:
           0.013265292 = score(doc=835,freq=3.0), product of:
             0.04432511 = queryWeight, product of:
               1.1471152 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.012229945 = queryNorm
             0.29927263 = fieldWeight in 835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.0078791985 = weight(abstract_txt:this in 835) [ClassicSimilarity], result of:
           0.0078791985 = score(doc=835,freq=3.0), product of:
             0.03447251 = queryWeight, product of:
               1.1681213 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.012229945 = queryNorm
             0.2285647 = fieldWeight in 835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.009530259 = weight(abstract_txt:also in 835) [ClassicSimilarity], result of:
           0.009530259 = score(doc=835,freq=1.0), product of:
             0.051280096 = queryWeight, product of:
               1.2338334 = boost
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.012229945 = queryNorm
             0.18584713 = fieldWeight in 835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.025989475 = weight(abstract_txt:study in 835) [ClassicSimilarity], result of:
           0.025989475 = score(doc=835,freq=4.0), product of:
             0.069401726 = queryWeight, product of:
               1.6574354 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.012229945 = queryNorm
             0.3744788 = fieldWeight in 835, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.022337126 = weight(abstract_txt:research in 835) [ClassicSimilarity], result of:
           0.022337126 = score(doc=835,freq=3.0), product of:
             0.074382894 = queryWeight, product of:
               1.9184171 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.012229945 = queryNorm
             0.30029923 = fieldWeight in 835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.06988773 = weight(abstract_txt:factors in 835) [ClassicSimilarity], result of:
           0.06988773 = score(doc=835,freq=2.0), product of:
             0.18214582 = queryWeight, product of:
               3.0020375 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.012229945 = queryNorm
             0.38369107 = fieldWeight in 835, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.28200096 = weight(abstract_txt:assessors in 835) [ClassicSimilarity], result of:
           0.28200096 = score(doc=835,freq=3.0), product of:
             0.34015146 = queryWeight, product of:
               3.1777394 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.012229945 = queryNorm
             0.82904524 = fieldWeight in 835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.19472927 = weight(abstract_txt:trec in 835) [ClassicSimilarity], result of:
           0.19472927 = score(doc=835,freq=4.0), product of:
             0.26574153 = queryWeight, product of:
               3.2432559 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.012229945 = queryNorm
             0.73277694 = fieldWeight in 835, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.11002855 = weight(abstract_txt:track in 835) [ClassicSimilarity], result of:
           0.11002855 = score(doc=835,freq=1.0), product of:
             0.28831166 = queryWeight, product of:
               3.3781786 = boost
               6.9783883 = idf(docFreq=111, maxDocs=44218)
               0.012229945 = queryNorm
             0.3816306 = fieldWeight in 835, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9783883 = idf(docFreq=111, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
         0.20181385 = weight(abstract_txt:relevance in 835) [ClassicSimilarity], result of:
           0.20181385 = score(doc=835,freq=3.0), product of:
             0.4320088 = queryWeight, product of:
               7.1623974 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012229945 = queryNorm
             0.46715218 = fieldWeight in 835, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=835)
       0.52 = coord(13/25)
   

2. Xu, Y.C..; Chen, Z.: Relevance judgment : what do information users consider beyond topicality? (2006) 0.42

   0.41755855 = sum of:
     0.41755855 = product of:
       1.3048705 = sum of:
         0.02498079 = weight(abstract_txt:implications in 5073) [ClassicSimilarity], result of:
           0.02498079 = score(doc=5073,freq=1.0), product of:
             0.059456218 = queryWeight, product of:
               1.0847636 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.012229945 = queryNorm
             0.4201544 = fieldWeight in 5073, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.011028581 = weight(abstract_txt:this in 5073) [ClassicSimilarity], result of:
           0.011028581 = score(doc=5073,freq=2.0), product of:
             0.03447251 = queryWeight, product of:
               1.1681213 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.012229945 = queryNorm
             0.31992394 = fieldWeight in 5073, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.016337587 = weight(abstract_txt:also in 5073) [ClassicSimilarity], result of:
           0.016337587 = score(doc=5073,freq=1.0), product of:
             0.051280096 = queryWeight, product of:
               1.2338334 = boost
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.012229945 = queryNorm
             0.31859508 = fieldWeight in 5073, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.031503998 = weight(abstract_txt:study in 5073) [ClassicSimilarity], result of:
           0.031503998 = score(doc=5073,freq=2.0), product of:
             0.069401726 = queryWeight, product of:
               1.6574354 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.012229945 = queryNorm
             0.45393682 = fieldWeight in 5073, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.27117607 = weight(abstract_txt:topicality in 5073) [ClassicSimilarity], result of:
           0.27117607 = score(doc=5073,freq=3.0), product of:
             0.20210962 = queryWeight, product of:
               2.0 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.012229945 = queryNorm
             1.3417277 = fieldWeight in 5073, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.084716715 = weight(abstract_txt:factors in 5073) [ClassicSimilarity], result of:
           0.084716715 = score(doc=5073,freq=1.0), product of:
             0.18214582 = queryWeight, product of:
               3.0020375 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.012229945 = queryNorm
             0.4651038 = fieldWeight in 5073, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.46563897 = weight(abstract_txt:judgment in 5073) [ClassicSimilarity], result of:
           0.46563897 = score(doc=5073,freq=2.0), product of:
             0.4784719 = queryWeight, product of:
               5.329977 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.012229945 = queryNorm
             0.97317934 = fieldWeight in 5073, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
         0.39948782 = weight(abstract_txt:relevance in 5073) [ClassicSimilarity], result of:
           0.39948782 = score(doc=5073,freq=4.0), product of:
             0.4320088 = queryWeight, product of:
               7.1623974 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012229945 = queryNorm
             0.9247215 = fieldWeight in 5073, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.09375 = fieldNorm(doc=5073)
       0.32 = coord(8/25)
   

3. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.40

   0.39591974 = sum of:
     0.39591974 = product of:
       1.4139991 = sum of:
         0.009004799 = weight(abstract_txt:this in 2591) [ClassicSimilarity], result of:
           0.009004799 = score(doc=2591,freq=3.0), product of:
             0.03447251 = queryWeight, product of:
               1.1681213 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.012229945 = queryNorm
             0.2612168 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.025722908 = weight(abstract_txt:study in 2591) [ClassicSimilarity], result of:
           0.025722908 = score(doc=2591,freq=3.0), product of:
             0.069401726 = queryWeight, product of:
               1.6574354 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.012229945 = queryNorm
             0.37063786 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.056477807 = weight(abstract_txt:factors in 2591) [ClassicSimilarity], result of:
           0.056477807 = score(doc=2591,freq=1.0), product of:
             0.18214582 = queryWeight, product of:
               3.0020375 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.012229945 = queryNorm
             0.3100692 = fieldWeight in 2591, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.32228678 = weight(abstract_txt:assessors in 2591) [ClassicSimilarity], result of:
           0.32228678 = score(doc=2591,freq=3.0), product of:
             0.34015146 = queryWeight, product of:
               3.1777394 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.012229945 = queryNorm
             0.94748026 = fieldWeight in 2591, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.15736501 = weight(abstract_txt:trec in 2591) [ClassicSimilarity], result of:
           0.15736501 = score(doc=2591,freq=2.0), product of:
             0.26574153 = queryWeight, product of:
               3.2432559 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.012229945 = queryNorm
             0.5921732 = fieldWeight in 2591, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.49082658 = weight(abstract_txt:judgment in 2591) [ClassicSimilarity], result of:
           0.49082658 = score(doc=2591,freq=5.0), product of:
             0.4784719 = queryWeight, product of:
               5.329977 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.012229945 = queryNorm
             1.0258211 = fieldWeight in 2591, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
         0.35231513 = weight(abstract_txt:relevance in 2591) [ClassicSimilarity], result of:
           0.35231513 = score(doc=2591,freq=7.0), product of:
             0.4320088 = queryWeight, product of:
               7.1623974 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012229945 = queryNorm
             0.8155277 = fieldWeight in 2591, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=2591)
       0.28 = coord(7/25)
   

4. Xu, Y.; Yin, H.: Novelty and topicality in interactive information retrieval (2008) 0.38

   0.37508887 = sum of:
     0.37508887 = product of:
       1.1721528 = sum of:
         0.009004799 = weight(abstract_txt:this in 1355) [ClassicSimilarity], result of:
           0.009004799 = score(doc=1355,freq=3.0), product of:
             0.03447251 = queryWeight, product of:
               1.1681213 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.012229945 = queryNorm
             0.2612168 = fieldWeight in 1355, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.010891724 = weight(abstract_txt:also in 1355) [ClassicSimilarity], result of:
           0.010891724 = score(doc=1355,freq=1.0), product of:
             0.051280096 = queryWeight, product of:
               1.2338334 = boost
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.012229945 = queryNorm
             0.21239673 = fieldWeight in 1355, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.029702256 = weight(abstract_txt:study in 1355) [ClassicSimilarity], result of:
           0.029702256 = score(doc=1355,freq=4.0), product of:
             0.069401726 = queryWeight, product of:
               1.6574354 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.012229945 = queryNorm
             0.42797574 = fieldWeight in 1355, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.014738682 = weight(abstract_txt:research in 1355) [ClassicSimilarity], result of:
           0.014738682 = score(doc=1355,freq=1.0), product of:
             0.074382894 = queryWeight, product of:
               1.9184171 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.012229945 = queryNorm
             0.19814612 = fieldWeight in 1355, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.20875145 = weight(abstract_txt:topicality in 1355) [ClassicSimilarity], result of:
           0.20875145 = score(doc=1355,freq=4.0), product of:
             0.20210962 = queryWeight, product of:
               2.0 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.012229945 = queryNorm
             1.0328625 = fieldWeight in 1355, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.047567487 = weight(abstract_txt:interactive in 1355) [ClassicSimilarity], result of:
           0.047567487 = score(doc=1355,freq=1.0), product of:
             0.13701177 = queryWeight, product of:
               2.0167918 = boost
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.012229945 = queryNorm
             0.3471781 = fieldWeight in 1355, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.620852 = weight(abstract_txt:judgment in 1355) [ClassicSimilarity], result of:
           0.620852 = score(doc=1355,freq=8.0), product of:
             0.4784719 = queryWeight, product of:
               5.329977 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.012229945 = queryNorm
             1.2975725 = fieldWeight in 1355, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
         0.2306444 = weight(abstract_txt:relevance in 1355) [ClassicSimilarity], result of:
           0.2306444 = score(doc=1355,freq=3.0), product of:
             0.4320088 = queryWeight, product of:
               7.1623974 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012229945 = queryNorm
             0.5338882 = fieldWeight in 1355, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=1355)
       0.32 = coord(8/25)
   

5. Spink, A.; Greisdorf, H.: Regions and levels : Measuring and mapping users' relevance judgements (2001) 0.37

   0.37350956 = sum of:
     0.37350956 = product of:
       0.9337739 = sum of:
         0.016653862 = weight(abstract_txt:implications in 5586) [ClassicSimilarity], result of:
           0.016653862 = score(doc=5586,freq=1.0), product of:
             0.059456218 = queryWeight, product of:
               1.0847636 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.012229945 = queryNorm
             0.28010294 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.008752822 = weight(abstract_txt:their in 5586) [ClassicSimilarity], result of:
           0.008752822 = score(doc=5586,freq=1.0), product of:
             0.04432511 = queryWeight, product of:
               1.1471152 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.012229945 = queryNorm
             0.19746871 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.0051989234 = weight(abstract_txt:this in 5586) [ClassicSimilarity], result of:
           0.0051989234 = score(doc=5586,freq=1.0), product of:
             0.03447251 = queryWeight, product of:
               1.1681213 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.012229945 = queryNorm
             0.1508136 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.010891724 = weight(abstract_txt:also in 5586) [ClassicSimilarity], result of:
           0.010891724 = score(doc=5586,freq=1.0), product of:
             0.051280096 = queryWeight, product of:
               1.2338334 = boost
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.012229945 = queryNorm
             0.21239673 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3983476 = idf(docFreq=4017, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.03440622 = weight(abstract_txt:retrieved in 5586) [ClassicSimilarity], result of:
           0.03440622 = score(doc=5586,freq=1.0), product of:
             0.096444756 = queryWeight, product of:
               1.3815793 = boost
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.012229945 = queryNorm
             0.35674536 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.021002667 = weight(abstract_txt:study in 5586) [ClassicSimilarity], result of:
           0.021002667 = score(doc=5586,freq=2.0), product of:
             0.069401726 = queryWeight, product of:
               1.6574354 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.012229945 = queryNorm
             0.30262455 = fieldWeight in 5586, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.014738682 = weight(abstract_txt:research in 5586) [ClassicSimilarity], result of:
           0.014738682 = score(doc=5586,freq=1.0), product of:
             0.074382894 = queryWeight, product of:
               1.9184171 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.012229945 = queryNorm
             0.19814612 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.10437573 = weight(abstract_txt:topicality in 5586) [ClassicSimilarity], result of:
           0.10437573 = score(doc=5586,freq=1.0), product of:
             0.20210962 = queryWeight, product of:
               2.0 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.012229945 = queryNorm
             0.5164313 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.21950433 = weight(abstract_txt:judgment in 5586) [ClassicSimilarity], result of:
           0.21950433 = score(doc=5586,freq=1.0), product of:
             0.4784719 = queryWeight, product of:
               5.329977 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.012229945 = queryNorm
             0.45876116 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.4982489 = weight(abstract_txt:relevance in 5586) [ClassicSimilarity], result of:
           0.4982489 = score(doc=5586,freq=14.0), product of:
             0.4320088 = queryWeight, product of:
               7.1623974 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.012229945 = queryNorm
             1.1533304 = fieldWeight in 5586, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
       0.4 = coord(10/25)