Document (#39593)

Author
Ravana, S.D.
Rajagopal, P.
Balakrishnan, V.
Title
Ranking retrieval systems using pseudo relevance judgments
Source
Aslib journal of information management. 67(2015) no.6, S.700-714
Year
2015
Abstract
Purpose In a system-based approach, replicating the web would require large test collections, and judging the relevancy of all documents per topic in creating relevance judgment through human assessors is infeasible. Due to the large amount of documents that requires judgment, there are possible errors introduced by human assessors because of disagreements. The paper aims to discuss these issues. Design/methodology/approach This study explores exponential variation and document ranking methods that generate a reliable set of relevance judgments (pseudo relevance judgments) to reduce human efforts. These methods overcome problems with large amounts of documents for judgment while avoiding human disagreement errors during the judgment process. This study utilizes two key factors: number of occurrences of each document per topic from all the system runs; and document rankings to generate the alternate methods. Findings The effectiveness of the proposed method is evaluated using the correlation coefficient of ranked systems using mean average precision scores between the original Text REtrieval Conference (TREC) relevance judgments and pseudo relevance judgments. The results suggest that the proposed document ranking method with a pool depth of 100 could be a reliable alternative to reduce human effort and disagreement errors involved in generating TREC-like relevance judgments. Originality/value Simple methods proposed in this study show improvement in the correlation coefficient in generating alternate relevance judgment without human assessors while contributing to information retrieval evaluation.
Content
Vgl.: http://dx.doi.org/10.1108/AJIM-03-2015-0046.
Theme
Retrievalalgorithmen

Similar documents (author)

  1. Balakrishnan, U.: DFG-Projekt: Coli-conc : das Mapping Tool "Cocoda" (2016) 5.91
    5.9096622 = sum of:
      5.9096622 = weight(author_txt:balakrishnan in 37) [ClassicSimilarity], result of:
        5.9096622 = fieldWeight in 37, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.45546 = idf(docFreq=8, maxDocs=42306)
          0.625 = fieldNorm(doc=37)
    
  2. Balakrishnan, U.; Voß, J.: ¬The Cocoda mapping tool (2015) 4.73
    4.72773 = sum of:
      4.72773 = weight(author_txt:balakrishnan in 1124) [ClassicSimilarity], result of:
        4.72773 = fieldWeight in 1124, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.45546 = idf(docFreq=8, maxDocs=42306)
          0.5 = fieldNorm(doc=1124)
    
  3. Balakrishnan, U.; Krausz, A,; Voss, J.: Cocoda - ein Konkordanztool für bibliothekarische Klassifikationssysteme (2015) 3.55
    3.5457973 = sum of:
      3.5457973 = weight(author_txt:balakrishnan in 4031) [ClassicSimilarity], result of:
        3.5457973 = fieldWeight in 4031, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.45546 = idf(docFreq=8, maxDocs=42306)
          0.375 = fieldNorm(doc=4031)
    
  4. Balakrishnan, V.; Ahmadi, K.; Ravana, S.D.: Improving retrieval relevance using users' explicit feedback (2016) 3.55
    3.5457973 = sum of:
      3.5457973 = weight(author_txt:balakrishnan in 4922) [ClassicSimilarity], result of:
        3.5457973 = fieldWeight in 4922, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.45546 = idf(docFreq=8, maxDocs=42306)
          0.375 = fieldNorm(doc=4922)
    
  5. Balakrishnan, U.; Voß, J.; Soergel, D.: Towards integrated systems for KOS management, mapping, and access : Coli-conc and its collaborative computer-assisted KOS mapping tool Cocoda (2018) 3.55
    3.5457973 = sum of:
      3.5457973 = weight(author_txt:balakrishnan in 1744) [ClassicSimilarity], result of:
        3.5457973 = fieldWeight in 1744, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          9.45546 = idf(docFreq=8, maxDocs=42306)
          0.375 = fieldNorm(doc=1744)
    

Similar documents (content)

  1. Rajagopal, P.; Ravana, S.D.; Koh, Y.S.; Balakrishnan, V.: Evaluating the effectiveness of information retrieval systems using effort-based relevance judgment (2019) 0.46
    0.46086287 = sum of:
      0.46086287 = product of:
        1.0474156 = sum of:
          0.019909939 = weight(abstract_txt:retrieval in 2205) [ClassicSimilarity], result of:
            0.019909939 = score(doc=2205,freq=3.0), product of:
              0.053116087 = queryWeight, product of:
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.015339911 = queryNorm
              0.3748382 = fieldWeight in 2205, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.011725686 = weight(abstract_txt:study in 2205) [ClassicSimilarity], result of:
            0.011725686 = score(doc=2205,freq=1.0), product of:
              0.05382434 = queryWeight, product of:
                1.006645 = boost
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.015339911 = queryNorm
              0.217851 = fieldWeight in 2205, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.02032935 = weight(abstract_txt:using in 2205) [ClassicSimilarity], result of:
            0.02032935 = score(doc=2205,freq=3.0), product of:
              0.053859435 = queryWeight, product of:
                1.0069731 = boost
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.015339911 = queryNorm
              0.377452 = fieldWeight in 2205, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.038608983 = weight(abstract_txt:documents in 2205) [ClassicSimilarity], result of:
            0.038608983 = score(doc=2205,freq=4.0), product of:
              0.07504564 = queryWeight, product of:
                1.1886382 = boost
                4.115787 = idf(docFreq=1875, maxDocs=42306)
                0.015339911 = queryNorm
              0.5144734 = fieldWeight in 2205, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.115787 = idf(docFreq=1875, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.05846205 = weight(abstract_txt:generating in 2205) [ClassicSimilarity], result of:
            0.05846205 = score(doc=2205,freq=1.0), product of:
              0.13722676 = queryWeight, product of:
                1.3123835 = boost
                6.8164024 = idf(docFreq=125, maxDocs=42306)
                0.015339911 = queryNorm
              0.42602515 = fieldWeight in 2205, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.8164024 = idf(docFreq=125, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.028029913 = weight(abstract_txt:proposed in 2205) [ClassicSimilarity], result of:
            0.028029913 = score(doc=2205,freq=1.0), product of:
              0.09622791 = queryWeight, product of:
                1.3459766 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.015339911 = queryNorm
              0.29128674 = fieldWeight in 2205, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.040914245 = weight(abstract_txt:document in 2205) [ClassicSimilarity], result of:
            0.040914245 = score(doc=2205,freq=2.0), product of:
              0.10816974 = queryWeight, product of:
                1.647818 = boost
                4.2793097 = idf(docFreq=1592, maxDocs=42306)
                0.015339911 = queryNorm
              0.37824112 = fieldWeight in 2205, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.2793097 = idf(docFreq=1592, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.059271168 = weight(abstract_txt:human in 2205) [ClassicSimilarity], result of:
            0.059271168 = score(doc=2205,freq=1.0), product of:
              0.19973715 = queryWeight, product of:
                2.7424037 = boost
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.015339911 = queryNorm
              0.29674584 = fieldWeight in 2205, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.18415163 = weight(abstract_txt:judgment in 2205) [ClassicSimilarity], result of:
            0.18415163 = score(doc=2205,freq=1.0), product of:
              0.40020868 = queryWeight, product of:
                3.543681 = boost
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.015339911 = queryNorm
              0.46013904 = fieldWeight in 2205, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.2181593 = weight(abstract_txt:relevance in 2205) [ClassicSimilarity], result of:
            0.2181593 = score(doc=2205,freq=6.0), product of:
              0.28840858 = queryWeight, product of:
                3.805182 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.015339911 = queryNorm
              0.7564245 = fieldWeight in 2205, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
          0.3678533 = weight(abstract_txt:judgments in 2205) [ClassicSimilarity], result of:
            0.3678533 = score(doc=2205,freq=4.0), product of:
              0.4249387 = queryWeight, product of:
                4.000047 = boost
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.015339911 = queryNorm
              0.86566204 = fieldWeight in 2205, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.0625 = fieldNorm(doc=2205)
        0.44 = coord(11/25)
    
  2. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.45
    0.44716227 = sum of:
      0.44716227 = product of:
        1.3973821 = sum of:
          0.023451371 = weight(abstract_txt:study in 1541) [ClassicSimilarity], result of:
            0.023451371 = score(doc=1541,freq=4.0), product of:
              0.05382434 = queryWeight, product of:
                1.006645 = boost
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.015339911 = queryNorm
              0.435702 = fieldWeight in 1541, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.011737156 = weight(abstract_txt:using in 1541) [ClassicSimilarity], result of:
            0.011737156 = score(doc=1541,freq=1.0), product of:
              0.053859435 = queryWeight, product of:
                1.0069731 = boost
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.015339911 = queryNorm
              0.217922 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.095128 = weight(abstract_txt:trec in 1541) [ClassicSimilarity], result of:
            0.095128 = score(doc=1541,freq=3.0), product of:
              0.13162991 = queryWeight, product of:
                1.2853419 = boost
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.015339911 = queryNorm
              0.72269285 = fieldWeight in 1541, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.31670305 = weight(abstract_txt:assessors in 1541) [ClassicSimilarity], result of:
            0.31670305 = score(doc=1541,freq=3.0), product of:
              0.33595487 = queryWeight, product of:
                2.514939 = boost
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.015339911 = queryNorm
              0.9426952 = fieldWeight in 1541, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.059271168 = weight(abstract_txt:human in 1541) [ClassicSimilarity], result of:
            0.059271168 = score(doc=1541,freq=1.0), product of:
              0.19973715 = queryWeight, product of:
                2.7424037 = boost
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.015339911 = queryNorm
              0.29674584 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.4117756 = weight(abstract_txt:judgment in 1541) [ClassicSimilarity], result of:
            0.4117756 = score(doc=1541,freq=5.0), product of:
              0.40020868 = queryWeight, product of:
                3.543681 = boost
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.015339911 = queryNorm
              1.0289022 = fieldWeight in 1541, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.2953891 = weight(abstract_txt:relevance in 1541) [ClassicSimilarity], result of:
            0.2953891 = score(doc=1541,freq=11.0), product of:
              0.28840858 = queryWeight, product of:
                3.805182 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.015339911 = queryNorm
              1.0242035 = fieldWeight in 1541, product of:
                3.3166249 = tf(freq=11.0), with freq of:
                  11.0 = termFreq=11.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.18392666 = weight(abstract_txt:judgments in 1541) [ClassicSimilarity], result of:
            0.18392666 = score(doc=1541,freq=1.0), product of:
              0.4249387 = queryWeight, product of:
                4.000047 = boost
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.015339911 = queryNorm
              0.43283102 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
        0.32 = coord(8/25)
    
  3. Losada, D.E.; Parapar, J.; Barreiro, A.: When to stop making relevance judgments? : a study of stopping methods for building information retrieval test collections (2019) 0.37
    0.37227866 = sum of:
      0.37227866 = product of:
        0.9306966 = sum of:
          0.011495009 = weight(abstract_txt:retrieval in 1593) [ClassicSimilarity], result of:
            0.011495009 = score(doc=1593,freq=1.0), product of:
              0.053116087 = queryWeight, product of:
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.015339911 = queryNorm
              0.21641295 = fieldWeight in 1593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.011737156 = weight(abstract_txt:using in 1593) [ClassicSimilarity], result of:
            0.011737156 = score(doc=1593,freq=1.0), product of:
              0.053859435 = queryWeight, product of:
                1.0069731 = boost
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.015339911 = queryNorm
              0.217922 = fieldWeight in 1593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.03343636 = weight(abstract_txt:documents in 1593) [ClassicSimilarity], result of:
            0.03343636 = score(doc=1593,freq=3.0), product of:
              0.07504564 = queryWeight, product of:
                1.1886382 = boost
                4.115787 = idf(docFreq=1875, maxDocs=42306)
                0.015339911 = queryNorm
              0.445547 = fieldWeight in 1593, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.115787 = idf(docFreq=1875, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.06620144 = weight(abstract_txt:reduce in 1593) [ClassicSimilarity], result of:
            0.06620144 = score(doc=1593,freq=2.0), product of:
              0.118328966 = queryWeight, product of:
                1.2186722 = boost
                6.3296742 = idf(docFreq=204, maxDocs=42306)
                0.015339911 = queryNorm
              0.55946946 = fieldWeight in 1593, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.3296742 = idf(docFreq=204, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.056704767 = weight(abstract_txt:reliable in 1593) [ClassicSimilarity], result of:
            0.056704767 = score(doc=1593,freq=1.0), product of:
              0.13446291 = queryWeight, product of:
                1.2991 = boost
                6.7474093 = idf(docFreq=134, maxDocs=42306)
                0.015339911 = queryNorm
              0.42171308 = fieldWeight in 1593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.7474093 = idf(docFreq=134, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.028029913 = weight(abstract_txt:proposed in 1593) [ClassicSimilarity], result of:
            0.028029913 = score(doc=1593,freq=1.0), product of:
              0.09622791 = queryWeight, product of:
                1.3459766 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.015339911 = queryNorm
              0.29128674 = fieldWeight in 1593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.06610809 = weight(abstract_txt:methods in 1593) [ClassicSimilarity], result of:
            0.06610809 = score(doc=1593,freq=6.0), product of:
              0.10327268 = queryWeight, product of:
                1.610086 = boost
                4.181321 = idf(docFreq=1756, maxDocs=42306)
                0.015339911 = queryNorm
              0.6401315 = fieldWeight in 1593, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                4.181321 = idf(docFreq=1756, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.18415163 = weight(abstract_txt:judgment in 1593) [ClassicSimilarity], result of:
            0.18415163 = score(doc=1593,freq=1.0), product of:
              0.40020868 = queryWeight, product of:
                3.543681 = boost
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.015339911 = queryNorm
              0.46013904 = fieldWeight in 1593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.15426192 = weight(abstract_txt:relevance in 1593) [ClassicSimilarity], result of:
            0.15426192 = score(doc=1593,freq=3.0), product of:
              0.28840858 = queryWeight, product of:
                3.805182 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.015339911 = queryNorm
              0.53487283 = fieldWeight in 1593, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
          0.3185703 = weight(abstract_txt:judgments in 1593) [ClassicSimilarity], result of:
            0.3185703 = score(doc=1593,freq=3.0), product of:
              0.4249387 = queryWeight, product of:
                4.000047 = boost
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.015339911 = queryNorm
              0.7496853 = fieldWeight in 1593, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.0625 = fieldNorm(doc=1593)
        0.4 = coord(10/25)
    
  4. Balatsoukas, P.; Ruthven, I.: ¬An eye-tracking approach to the analysis of relevance judgments on the Web : the case of Google search engine (2012) 0.30
    0.29776442 = sum of:
      0.29776442 = product of:
        0.93051386 = sum of:
          0.011495009 = weight(abstract_txt:retrieval in 2380) [ClassicSimilarity], result of:
            0.011495009 = score(doc=2380,freq=1.0), product of:
              0.053116087 = queryWeight, product of:
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.015339911 = queryNorm
              0.21641295 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.011725686 = weight(abstract_txt:study in 2380) [ClassicSimilarity], result of:
            0.011725686 = score(doc=2380,freq=1.0), product of:
              0.05382434 = queryWeight, product of:
                1.006645 = boost
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.015339911 = queryNorm
              0.217851 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.011737156 = weight(abstract_txt:using in 2380) [ClassicSimilarity], result of:
            0.011737156 = score(doc=2380,freq=1.0), product of:
              0.053859435 = queryWeight, product of:
                1.0069731 = boost
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.015339911 = queryNorm
              0.217922 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486752 = idf(docFreq=3518, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.04864096 = weight(abstract_txt:ranking in 2380) [ClassicSimilarity], result of:
            0.04864096 = score(doc=2380,freq=1.0), product of:
              0.13895941 = queryWeight, product of:
                1.6174504 = boost
                5.600595 = idf(docFreq=424, maxDocs=42306)
                0.015339911 = queryNorm
              0.3500372 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.600595 = idf(docFreq=424, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.059271168 = weight(abstract_txt:human in 2380) [ClassicSimilarity], result of:
            0.059271168 = score(doc=2380,freq=1.0), product of:
              0.19973715 = queryWeight, product of:
                2.7424037 = boost
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.015339911 = queryNorm
              0.29674584 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.18415163 = weight(abstract_txt:judgment in 2380) [ClassicSimilarity], result of:
            0.18415163 = score(doc=2380,freq=1.0), product of:
              0.40020868 = queryWeight, product of:
                3.543681 = boost
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.015339911 = queryNorm
              0.46013904 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.3622246 = idf(docFreq=72, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.23563896 = weight(abstract_txt:relevance in 2380) [ClassicSimilarity], result of:
            0.23563896 = score(doc=2380,freq=7.0), product of:
              0.28840858 = queryWeight, product of:
                3.805182 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.015339911 = queryNorm
              0.81703174 = fieldWeight in 2380, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
          0.3678533 = weight(abstract_txt:judgments in 2380) [ClassicSimilarity], result of:
            0.3678533 = score(doc=2380,freq=4.0), product of:
              0.4249387 = queryWeight, product of:
                4.000047 = boost
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.015339911 = queryNorm
              0.86566204 = fieldWeight in 2380, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.0625 = fieldNorm(doc=2380)
        0.32 = coord(8/25)
    
  5. Reichenbacher, T.; Sabbata, S. de; Purves, R.S.; Fabrikant, S.I.: Assessing geographic relevance for mobile search : a computational model and its validation via crowdsourcing (2016) 0.29
    0.29454333 = sum of:
      0.29454333 = product of:
        0.92044795 = sum of:
          0.020320496 = weight(abstract_txt:retrieval in 64) [ClassicSimilarity], result of:
            0.020320496 = score(doc=64,freq=2.0), product of:
              0.053116087 = queryWeight, product of:
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.015339911 = queryNorm
              0.38256764 = fieldWeight in 64, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.4626071 = idf(docFreq=3604, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.014657107 = weight(abstract_txt:study in 64) [ClassicSimilarity], result of:
            0.014657107 = score(doc=64,freq=1.0), product of:
              0.05382434 = queryWeight, product of:
                1.006645 = boost
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.015339911 = queryNorm
              0.27231374 = fieldWeight in 64, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.485616 = idf(docFreq=3522, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.024130614 = weight(abstract_txt:documents in 64) [ClassicSimilarity], result of:
            0.024130614 = score(doc=64,freq=1.0), product of:
              0.07504564 = queryWeight, product of:
                1.1886382 = boost
                4.115787 = idf(docFreq=1875, maxDocs=42306)
                0.015339911 = queryNorm
              0.32154587 = fieldWeight in 64, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.115787 = idf(docFreq=1875, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.075435184 = weight(abstract_txt:methods in 64) [ClassicSimilarity], result of:
            0.075435184 = score(doc=64,freq=5.0), product of:
              0.10327268 = queryWeight, product of:
                1.610086 = boost
                4.181321 = idf(docFreq=1756, maxDocs=42306)
                0.015339911 = queryNorm
              0.7304467 = fieldWeight in 64, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                4.181321 = idf(docFreq=1756, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.22856073 = weight(abstract_txt:assessors in 64) [ClassicSimilarity], result of:
            0.22856073 = score(doc=64,freq=1.0), product of:
              0.33595487 = queryWeight, product of:
                2.514939 = boost
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.015339911 = queryNorm
              0.68033165 = fieldWeight in 64, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.104777604 = weight(abstract_txt:human in 64) [ClassicSimilarity], result of:
            0.104777604 = score(doc=64,freq=2.0), product of:
              0.19973715 = queryWeight, product of:
                2.7424037 = boost
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.015339911 = queryNorm
              0.52457744 = fieldWeight in 64, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.7479334 = idf(docFreq=996, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.2226579 = weight(abstract_txt:relevance in 64) [ClassicSimilarity], result of:
            0.2226579 = score(doc=64,freq=4.0), product of:
              0.28840858 = queryWeight, product of:
                3.805182 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.015339911 = queryNorm
              0.7720225 = fieldWeight in 64, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
          0.2299083 = weight(abstract_txt:judgments in 64) [ClassicSimilarity], result of:
            0.2299083 = score(doc=64,freq=1.0), product of:
              0.4249387 = queryWeight, product of:
                4.000047 = boost
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.015339911 = queryNorm
              0.54103875 = fieldWeight in 64, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.9252963 = idf(docFreq=112, maxDocs=42306)
                0.078125 = fieldNorm(doc=64)
        0.32 = coord(8/25)