Document (#39592)

Author

Ravana, S.D.
Rajagopal, P.
Balakrishnan, V.

Title

Ranking retrieval systems using pseudo relevance judgments

Source

Aslib journal of information management. 67(2015) no.6, S.700-714

Year

2015

Abstract

Purpose In a system-based approach, replicating the web would require large test collections, and judging the relevancy of all documents per topic in creating relevance judgment through human assessors is infeasible. Due to the large amount of documents that requires judgment, there are possible errors introduced by human assessors because of disagreements. The paper aims to discuss these issues. Design/methodology/approach This study explores exponential variation and document ranking methods that generate a reliable set of relevance judgments (pseudo relevance judgments) to reduce human efforts. These methods overcome problems with large amounts of documents for judgment while avoiding human disagreement errors during the judgment process. This study utilizes two key factors: number of occurrences of each document per topic from all the system runs; and document rankings to generate the alternate methods. Findings The effectiveness of the proposed method is evaluated using the correlation coefficient of ranked systems using mean average precision scores between the original Text REtrieval Conference (TREC) relevance judgments and pseudo relevance judgments. The results suggest that the proposed document ranking method with a pool depth of 100 could be a reliable alternative to reduce human effort and disagreement errors involved in generating TREC-like relevance judgments. Originality/value Simple methods proposed in this study show improvement in the correlation coefficient in generating alternate relevance judgment without human assessors while contributing to information retrieval evaluation.

Content

Vgl.: http://dx.doi.org/10.1108/AJIM-03-2015-0046.

Theme

Retrievalalgorithmen

Similar documents (author)

1. Balakrishnan, U.: DFG-Projekt: Coli-conc : das Mapping Tool "Cocoda" (2016) 5.81

   5.81187 = sum of:
     5.81187 = weight(author_txt:balakrishnan in 3036) [ClassicSimilarity], result of:
       5.81187 = fieldWeight in 3036, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.298992 = idf(docFreq=10, maxDocs=44218)
         0.625 = fieldNorm(doc=3036)
   

2. Balakrishnan, U.; Voß, J.: ¬The Cocoda mapping tool (2015) 4.65

   4.649496 = sum of:
     4.649496 = weight(author_txt:balakrishnan in 4205) [ClassicSimilarity], result of:
       4.649496 = fieldWeight in 4205, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.298992 = idf(docFreq=10, maxDocs=44218)
         0.5 = fieldNorm(doc=4205)
   

3. Balakrishnan, U.; Krausz, A,; Voss, J.: Cocoda - ein Konkordanztool für bibliothekarische Klassifikationssysteme (2015) 3.49

   3.487122 = sum of:
     3.487122 = weight(author_txt:balakrishnan in 2030) [ClassicSimilarity], result of:
       3.487122 = fieldWeight in 2030, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.298992 = idf(docFreq=10, maxDocs=44218)
         0.375 = fieldNorm(doc=2030)
   

4. Balakrishnan, V.; Ahmadi, K.; Ravana, S.D.: Improving retrieval relevance using users' explicit feedback (2016) 3.49

   3.487122 = sum of:
     3.487122 = weight(author_txt:balakrishnan in 2921) [ClassicSimilarity], result of:
       3.487122 = fieldWeight in 2921, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.298992 = idf(docFreq=10, maxDocs=44218)
         0.375 = fieldNorm(doc=2921)
   

5. Balakrishnan, U.; Voß, J.; Soergel, D.: Towards integrated systems for KOS management, mapping, and access : Coli-conc and its collaborative computer-assisted KOS mapping tool Cocoda (2018) 3.49

   3.487122 = sum of:
     3.487122 = weight(author_txt:balakrishnan in 4825) [ClassicSimilarity], result of:
       3.487122 = fieldWeight in 4825, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.298992 = idf(docFreq=10, maxDocs=44218)
         0.375 = fieldNorm(doc=4825)
   

Similar documents (content)

1. Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019) 0.46

   0.45812514 = sum of:
     0.45812514 = product of:
       1.0411935 = sum of:
         0.011173657 = weight(abstract_txt:study in 5287) [ClassicSimilarity], result of:
           0.011173657 = score(doc=5287,freq=1.0), product of:
             0.05221631 = queryWeight, product of:
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.015250955 = queryNorm
             0.21398787 = fieldWeight in 5287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.020027678 = weight(abstract_txt:using in 5287) [ClassicSimilarity], result of:
           0.020027678 = score(doc=5287,freq=3.0), product of:
             0.053422302 = queryWeight, product of:
               1.0114821 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.015250955 = queryNorm
             0.37489358 = fieldWeight in 5287, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.020236956 = weight(abstract_txt:retrieval in 5287) [ClassicSimilarity], result of:
           0.020236956 = score(doc=5287,freq=3.0), product of:
             0.053793814 = queryWeight, product of:
               1.0149931 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015250955 = queryNorm
             0.37619486 = fieldWeight in 5287, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.038976375 = weight(abstract_txt:documents in 5287) [ClassicSimilarity], result of:
           0.038976375 = score(doc=5287,freq=4.0), product of:
             0.07565836 = queryWeight, product of:
               1.2037197 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.015250955 = queryNorm
             0.5151628 = fieldWeight in 5287, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.058142718 = weight(abstract_txt:generating in 5287) [ClassicSimilarity], result of:
           0.058142718 = score(doc=5287,freq=1.0), product of:
             0.13697535 = queryWeight, product of:
               1.3224294 = boost
               6.7916126 = idf(docFreq=134, maxDocs=44218)
               0.015250955 = queryNorm
             0.4244758 = fieldWeight in 5287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7916126 = idf(docFreq=134, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.027263189 = weight(abstract_txt:proposed in 5287) [ClassicSimilarity], result of:
           0.027263189 = score(doc=5287,freq=1.0), product of:
             0.09463687 = queryWeight, product of:
               1.3462542 = boost
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.015250955 = queryNorm
             0.2880821 = fieldWeight in 5287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.04152268 = weight(abstract_txt:document in 5287) [ClassicSimilarity], result of:
           0.04152268 = score(doc=5287,freq=2.0), product of:
             0.10943825 = queryWeight, product of:
               1.6716712 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.015250955 = queryNorm
             0.37941656 = fieldWeight in 5287, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.057513967 = weight(abstract_txt:human in 5287) [ClassicSimilarity], result of:
           0.057513967 = score(doc=5287,freq=1.0), product of:
             0.19612585 = queryWeight, product of:
               2.7408125 = boost
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.015250955 = queryNorm
             0.29325032 = fieldWeight in 5287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.1835002 = weight(abstract_txt:judgment in 5287) [ClassicSimilarity], result of:
           0.1835002 = score(doc=5287,freq=1.0), product of:
             0.3999907 = queryWeight, product of:
               3.573109 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.015250955 = queryNorm
             0.45876116 = fieldWeight in 5287, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.21814303 = weight(abstract_txt:relevance in 5287) [ClassicSimilarity], result of:
           0.21814303 = score(doc=5287,freq=6.0), product of:
             0.2889189 = queryWeight, product of:
               3.8412209 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015250955 = queryNorm
             0.755032 = fieldWeight in 5287, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
         0.36469302 = weight(abstract_txt:judgments in 5287) [ClassicSimilarity], result of:
           0.36469302 = score(doc=5287,freq=4.0), product of:
             0.4232701 = queryWeight, product of:
               4.0264354 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.015250955 = queryNorm
             0.86160827 = fieldWeight in 5287, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=5287)
       0.44 = coord(11/25)
   

2. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.45

   0.44781974 = sum of:
     0.44781974 = product of:
       1.3994367 = sum of:
         0.022347314 = weight(abstract_txt:study in 4540) [ClassicSimilarity], result of:
           0.022347314 = score(doc=4540,freq=4.0), product of:
             0.05221631 = queryWeight, product of:
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.015250955 = queryNorm
             0.42797574 = fieldWeight in 4540, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.011562985 = weight(abstract_txt:using in 4540) [ClassicSimilarity], result of:
           0.011562985 = score(doc=4540,freq=1.0), product of:
             0.053422302 = queryWeight, product of:
               1.0114821 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.015250955 = queryNorm
             0.21644491 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.096671514 = weight(abstract_txt:trec in 4540) [ClassicSimilarity], result of:
           0.096671514 = score(doc=4540,freq=3.0), product of:
             0.13329202 = queryWeight, product of:
               1.3045279 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.015250955 = queryNorm
             0.7252611 = fieldWeight in 4540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.32330853 = weight(abstract_txt:assessors in 4540) [ClassicSimilarity], result of:
           0.32330853 = score(doc=4540,freq=3.0), product of:
             0.34122983 = queryWeight, product of:
               2.5563505 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.015250955 = queryNorm
             0.94748026 = fieldWeight in 4540, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.057513967 = weight(abstract_txt:human in 4540) [ClassicSimilarity], result of:
           0.057513967 = score(doc=4540,freq=1.0), product of:
             0.19612585 = queryWeight, product of:
               2.7408125 = boost
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.015250955 = queryNorm
             0.29325032 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.4103189 = weight(abstract_txt:judgment in 4540) [ClassicSimilarity], result of:
           0.4103189 = score(doc=4540,freq=5.0), product of:
             0.3999907 = queryWeight, product of:
               3.573109 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.015250955 = queryNorm
             1.0258211 = fieldWeight in 4540, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.29536703 = weight(abstract_txt:relevance in 4540) [ClassicSimilarity], result of:
           0.29536703 = score(doc=4540,freq=11.0), product of:
             0.2889189 = queryWeight, product of:
               3.8412209 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015250955 = queryNorm
             1.0223181 = fieldWeight in 4540, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
         0.18234651 = weight(abstract_txt:judgments in 4540) [ClassicSimilarity], result of:
           0.18234651 = score(doc=4540,freq=1.0), product of:
             0.4232701 = queryWeight, product of:
               4.0264354 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.015250955 = queryNorm
             0.43080413 = fieldWeight in 4540, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=4540)
       0.32 = coord(8/25)
   

3. Losada, D.E.; Parapar, J.; Barreiro, A.: When to stop making relevance judgments? : a study of stopping methods for building information retrieval test collections (2019) 0.37

   0.36972818 = sum of:
     0.36972818 = product of:
       0.92432046 = sum of:
         0.011562985 = weight(abstract_txt:using in 4674) [ClassicSimilarity], result of:
           0.011562985 = score(doc=4674,freq=1.0), product of:
             0.053422302 = queryWeight, product of:
               1.0114821 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.015250955 = queryNorm
             0.21644491 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.011683812 = weight(abstract_txt:retrieval in 4674) [ClassicSimilarity], result of:
           0.011683812 = score(doc=4674,freq=1.0), product of:
             0.053793814 = queryWeight, product of:
               1.0149931 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015250955 = queryNorm
             0.21719621 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.033754528 = weight(abstract_txt:documents in 4674) [ClassicSimilarity], result of:
           0.033754528 = score(doc=4674,freq=3.0), product of:
             0.07565836 = queryWeight, product of:
               1.2037197 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.015250955 = queryNorm
             0.44614407 = fieldWeight in 4674, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.06531075 = weight(abstract_txt:reduce in 4674) [ClassicSimilarity], result of:
           0.06531075 = score(doc=4674,freq=2.0), product of:
             0.117478564 = queryWeight, product of:
               1.2247025 = boost
               6.2897153 = idf(docFreq=222, maxDocs=44218)
               0.015250955 = queryNorm
             0.5559375 = fieldWeight in 4674, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.2897153 = idf(docFreq=222, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.05632669 = weight(abstract_txt:reliable in 4674) [ClassicSimilarity], result of:
           0.05632669 = score(doc=4674,freq=1.0), product of:
             0.13410811 = queryWeight, product of:
               1.3085153 = boost
               6.7201533 = idf(docFreq=144, maxDocs=44218)
               0.015250955 = queryNorm
             0.42000958 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7201533 = idf(docFreq=144, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.027263189 = weight(abstract_txt:proposed in 4674) [ClassicSimilarity], result of:
           0.027263189 = score(doc=4674,freq=1.0), product of:
             0.09463687 = queryWeight, product of:
               1.3462542 = boost
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.015250955 = queryNorm
             0.2880821 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.6093135 = idf(docFreq=1196, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.06483454 = weight(abstract_txt:methods in 4674) [ClassicSimilarity], result of:
           0.06483454 = score(doc=4674,freq=6.0), product of:
             0.10212751 = queryWeight, product of:
               1.6148703 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.015250955 = queryNorm
             0.6348392 = fieldWeight in 4674, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.1835002 = weight(abstract_txt:judgment in 4674) [ClassicSimilarity], result of:
           0.1835002 = score(doc=4674,freq=1.0), product of:
             0.3999907 = queryWeight, product of:
               3.573109 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.015250955 = queryNorm
             0.45876116 = fieldWeight in 4674, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.1542504 = weight(abstract_txt:relevance in 4674) [ClassicSimilarity], result of:
           0.1542504 = score(doc=4674,freq=3.0), product of:
             0.2889189 = queryWeight, product of:
               3.8412209 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015250955 = queryNorm
             0.5338882 = fieldWeight in 4674, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
         0.31583342 = weight(abstract_txt:judgments in 4674) [ClassicSimilarity], result of:
           0.31583342 = score(doc=4674,freq=3.0), product of:
             0.4232701 = queryWeight, product of:
               4.0264354 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.015250955 = queryNorm
             0.74617463 = fieldWeight in 4674, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=4674)
       0.4 = coord(10/25)
   

4. Balatsoukas, P.; Ruthven, I.: ¬An eye-tracking approach to the analysis of relevance judgments on the Web : the case of Google search engine (2012) 0.30

   0.29587498 = sum of:
     0.29587498 = product of:
       0.9246093 = sum of:
         0.011173657 = weight(abstract_txt:study in 379) [ClassicSimilarity], result of:
           0.011173657 = score(doc=379,freq=1.0), product of:
             0.05221631 = queryWeight, product of:
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.015250955 = queryNorm
             0.21398787 = fieldWeight in 379, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.011562985 = weight(abstract_txt:using in 379) [ClassicSimilarity], result of:
           0.011562985 = score(doc=379,freq=1.0), product of:
             0.053422302 = queryWeight, product of:
               1.0114821 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.015250955 = queryNorm
             0.21644491 = fieldWeight in 379, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.011683812 = weight(abstract_txt:retrieval in 379) [ClassicSimilarity], result of:
           0.011683812 = score(doc=379,freq=1.0), product of:
             0.053793814 = queryWeight, product of:
               1.0149931 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015250955 = queryNorm
             0.21719621 = fieldWeight in 379, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.048860274 = weight(abstract_txt:ranking in 379) [ClassicSimilarity], result of:
           0.048860274 = score(doc=379,freq=1.0), product of:
             0.13963038 = queryWeight, product of:
               1.6352602 = boost
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.015250955 = queryNorm
             0.34992582 = fieldWeight in 379, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.057513967 = weight(abstract_txt:human in 379) [ClassicSimilarity], result of:
           0.057513967 = score(doc=379,freq=1.0), product of:
             0.19612585 = queryWeight, product of:
               2.7408125 = boost
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.015250955 = queryNorm
             0.29325032 = fieldWeight in 379, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.1835002 = weight(abstract_txt:judgment in 379) [ClassicSimilarity], result of:
           0.1835002 = score(doc=379,freq=1.0), product of:
             0.3999907 = queryWeight, product of:
               3.573109 = boost
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.015250955 = queryNorm
             0.45876116 = fieldWeight in 379, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3401785 = idf(docFreq=77, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.23562136 = weight(abstract_txt:relevance in 379) [ClassicSimilarity], result of:
           0.23562136 = score(doc=379,freq=7.0), product of:
             0.2889189 = queryWeight, product of:
               3.8412209 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015250955 = queryNorm
             0.8155277 = fieldWeight in 379, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
         0.36469302 = weight(abstract_txt:judgments in 379) [ClassicSimilarity], result of:
           0.36469302 = score(doc=379,freq=4.0), product of:
             0.4232701 = queryWeight, product of:
               4.0264354 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.015250955 = queryNorm
             0.86160827 = fieldWeight in 379, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=379)
       0.32 = coord(8/25)
   

5. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 0.29

   0.29393187 = sum of:
     0.29393187 = product of:
       0.9185371 = sum of:
         0.013967072 = weight(abstract_txt:study in 3145) [ClassicSimilarity], result of:
           0.013967072 = score(doc=3145,freq=1.0), product of:
             0.05221631 = queryWeight, product of:
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.015250955 = queryNorm
             0.26748484 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.020654257 = weight(abstract_txt:retrieval in 3145) [ClassicSimilarity], result of:
           0.020654257 = score(doc=3145,freq=2.0), product of:
             0.053793814 = queryWeight, product of:
               1.0149931 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015250955 = queryNorm
             0.38395226 = fieldWeight in 3145, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.024360234 = weight(abstract_txt:documents in 3145) [ClassicSimilarity], result of:
           0.024360234 = score(doc=3145,freq=1.0), product of:
             0.07565836 = queryWeight, product of:
               1.2037197 = boost
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.015250955 = queryNorm
             0.32197678 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1213026 = idf(docFreq=1949, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.073981956 = weight(abstract_txt:methods in 3145) [ClassicSimilarity], result of:
           0.073981956 = score(doc=3145,freq=5.0), product of:
             0.10212751 = queryWeight, product of:
               1.6148703 = boost
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.015250955 = queryNorm
             0.72440773 = fieldWeight in 3145, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.146752 = idf(docFreq=1900, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.23332785 = weight(abstract_txt:assessors in 3145) [ClassicSimilarity], result of:
           0.23332785 = score(doc=3145,freq=1.0), product of:
             0.34122983 = queryWeight, product of:
               2.5563505 = boost
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.015250955 = queryNorm
             0.683785 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.752448 = idf(docFreq=18, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.10167129 = weight(abstract_txt:human in 3145) [ClassicSimilarity], result of:
           0.10167129 = score(doc=3145,freq=2.0), product of:
             0.19612585 = queryWeight, product of:
               2.7408125 = boost
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.015250955 = queryNorm
             0.5183982 = fieldWeight in 3145, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.692005 = idf(docFreq=1101, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.22264127 = weight(abstract_txt:relevance in 3145) [ClassicSimilarity], result of:
           0.22264127 = score(doc=3145,freq=4.0), product of:
             0.2889189 = queryWeight, product of:
               3.8412209 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015250955 = queryNorm
             0.7706013 = fieldWeight in 3145, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
         0.22793315 = weight(abstract_txt:judgments in 3145) [ClassicSimilarity], result of:
           0.22793315 = score(doc=3145,freq=1.0), product of:
             0.4232701 = queryWeight, product of:
               4.0264354 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.015250955 = queryNorm
             0.5385052 = fieldWeight in 3145, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.078125 = fieldNorm(doc=3145)
       0.32 = coord(8/25)