Document (#38784)

Author
Ruthven, I.
Title
Relevance behaviour in TREC
Source
Journal of documentation. 70(2014) no.6, S.1098-1117
Year
2014
Abstract
Purpose - The purpose of this paper is to examine how various types of TREC data can be used to better understand relevance and serve as test-bed for exploring relevance. The author proposes that there are many interesting studies that can be performed on the TREC data collections that are not directly related to evaluating systems but to learning more about human judgements of information and relevance and that these studies can provide useful research questions for other types of investigation. Design/methodology/approach - Through several case studies the author shows how existing data from TREC can be used to learn more about the factors that may affect relevance judgements and interactive search decisions and answer new research questions for exploring relevance. Findings - The paper uncovers factors, such as familiarity, interest and strictness of relevance criteria, that affect the nature of relevance assessments within TREC, contrasting these against findings from user studies of relevance. Research limitations/implications - The research only considers certain uses of TREC data and assessment given by professional relevance assessors but motivates further exploration of the TREC data so that the research community can further exploit the effort involved in the construction of TREC test collections. Originality/value - The paper presents an original viewpoint on relevance investigations and TREC itself by motivating TREC as a source of inspiration on understanding relevance rather than purely as a source of evaluation material.
Content
Beitrag in einem Special Issue: Festschrift in honour of Nigel Ford
Theme
Retrievalstudien
Object
TREC

Similar documents (author)

  1. Ruthven, R.: Interactive information retrieval (2008) 5.15
    5.1548843 = sum of:
      5.1548843 = weight(author_txt:ruthven in 2839) [ClassicSimilarity], result of:
        5.1548843 = fieldWeight in 2839, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.247815 = idf(docFreq=30, maxDocs=43556)
          0.625 = fieldNorm(doc=2839)
    
  2. Ruthven, I.: Integrating approaches to relevance (2005) 5.15
    5.1548843 = sum of:
      5.1548843 = weight(author_txt:ruthven in 1761) [ClassicSimilarity], result of:
        5.1548843 = fieldWeight in 1761, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.247815 = idf(docFreq=30, maxDocs=43556)
          0.625 = fieldNorm(doc=1761)
    
  3. Ruthven, I.: ¬The language of information need : differentiating conscious and formalized information needs (2019) 5.15
    5.1548843 = sum of:
      5.1548843 = weight(author_txt:ruthven in 1321) [ClassicSimilarity], result of:
        5.1548843 = fieldWeight in 1321, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.247815 = idf(docFreq=30, maxDocs=43556)
          0.625 = fieldNorm(doc=1321)
    
  4. Ruthven, I.: Resonance and the experience of relevance (2021) 5.15
    5.1548843 = sum of:
      5.1548843 = weight(author_txt:ruthven in 2498) [ClassicSimilarity], result of:
        5.1548843 = fieldWeight in 2498, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.247815 = idf(docFreq=30, maxDocs=43556)
          0.625 = fieldNorm(doc=2498)
    
  5. Ruthven, I.: ¬An information behavior theory of transitions (2022) 5.15
    5.1548843 = sum of:
      5.1548843 = weight(author_txt:ruthven in 2817) [ClassicSimilarity], result of:
        5.1548843 = fieldWeight in 2817, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.247815 = idf(docFreq=30, maxDocs=43556)
          0.625 = fieldNorm(doc=2817)
    

Similar documents (content)

  1. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.60
    0.6019712 = sum of:
      0.6019712 = product of:
        1.3681164 = sum of:
          0.014953824 = weight(abstract_txt:findings in 1538) [ClassicSimilarity], result of:
            0.014953824 = score(doc=1538,freq=1.0), product of:
              0.057191808 = queryWeight, product of:
                1.0183673 = boost
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.013424278 = queryNorm
              0.26146793 = fieldWeight in 1538, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.117980115 = weight(abstract_txt:assessors in 1538) [ClassicSimilarity], result of:
            0.117980115 = score(doc=1538,freq=3.0), product of:
              0.12473484 = queryWeight, product of:
                1.0634484 = boost
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.013424278 = queryNorm
              0.94584733 = fieldWeight in 1538, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.018381111 = weight(abstract_txt:purpose in 1538) [ClassicSimilarity], result of:
            0.018381111 = score(doc=1538,freq=1.0), product of:
              0.065626636 = queryWeight, product of:
                1.0908816 = boost
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.013424278 = queryNorm
              0.28008613 = fieldWeight in 1538, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.050369956 = weight(abstract_txt:factors in 1538) [ClassicSimilarity], result of:
            0.050369956 = score(doc=1538,freq=4.0), product of:
              0.080958046 = queryWeight, product of:
                1.2116231 = boost
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.013424278 = queryNorm
              0.62217355 = fieldWeight in 1538, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.042988364 = weight(abstract_txt:affect in 1538) [ClassicSimilarity], result of:
            0.042988364 = score(doc=1538,freq=1.0), product of:
              0.11562886 = queryWeight, product of:
                1.448007 = boost
                5.9484615 = idf(docFreq=308, maxDocs=43556)
                0.013424278 = queryNorm
              0.37177885 = fieldWeight in 1538, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.9484615 = idf(docFreq=308, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.037038475 = weight(abstract_txt:research in 1538) [ClassicSimilarity], result of:
            0.037038475 = score(doc=1538,freq=5.0), product of:
              0.08309809 = queryWeight, product of:
                1.9408995 = boost
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.013424278 = queryNorm
              0.44571993 = fieldWeight in 1538, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.016727613 = weight(abstract_txt:that in 1538) [ClassicSimilarity], result of:
            0.016727613 = score(doc=1538,freq=3.0), product of:
              0.06487896 = queryWeight, product of:
                2.0291936 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.013424278 = queryNorm
              0.257828 = fieldWeight in 1538, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.019262029 = weight(abstract_txt:data in 1538) [ClassicSimilarity], result of:
            0.019262029 = score(doc=1538,freq=1.0), product of:
              0.091892555 = queryWeight, product of:
                2.041022 = boost
                3.3538349 = idf(docFreq=4137, maxDocs=43556)
                0.013424278 = queryNorm
              0.20961468 = fieldWeight in 1538, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.3538349 = idf(docFreq=4137, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.032245513 = weight(abstract_txt:studies in 1538) [ClassicSimilarity], result of:
            0.032245513 = score(doc=1538,freq=1.0), product of:
              0.12026941 = queryWeight, product of:
                2.088479 = boost
                4.289771 = idf(docFreq=1622, maxDocs=43556)
                0.013424278 = queryNorm
              0.2681107 = fieldWeight in 1538, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.289771 = idf(docFreq=1622, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.4882464 = weight(abstract_txt:relevance in 1538) [ClassicSimilarity], result of:
            0.4882464 = score(doc=1538,freq=11.0), product of:
              0.47735834 = queryWeight, product of:
                7.206683 = boost
                4.934216 = idf(docFreq=851, maxDocs=43556)
                0.013424278 = queryNorm
              1.022809 = fieldWeight in 1538, product of:
                3.3166249 = tf(freq=11.0), with freq of:
                  11.0 = termFreq=11.0
                4.934216 = idf(docFreq=851, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
          0.5299229 = weight(abstract_txt:trec in 1538) [ClassicSimilarity], result of:
            0.5299229 = score(doc=1538,freq=3.0), product of:
              0.7315719 = queryWeight, product of:
                8.144249 = boost
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.013424278 = queryNorm
              0.724362 = fieldWeight in 1538, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.0625 = fieldNorm(doc=1538)
        0.44 = coord(11/25)
    
  2. Ruthven, I.; Baillie, M.; Elsweiler, D.: ¬The relative effects of knowledge, interest and confidence in assessing relevance (2007) 0.55
    0.5490018 = sum of:
      0.5490018 = product of:
        1.1437538 = sum of:
          0.013084596 = weight(abstract_txt:findings in 2833) [ClassicSimilarity], result of:
            0.013084596 = score(doc=2833,freq=1.0), product of:
              0.057191808 = queryWeight, product of:
                1.0183673 = boost
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.013424278 = queryNorm
              0.22878444 = fieldWeight in 2833, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.1032326 = weight(abstract_txt:assessors in 2833) [ClassicSimilarity], result of:
            0.1032326 = score(doc=2833,freq=3.0), product of:
              0.12473484 = queryWeight, product of:
                1.0634484 = boost
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.013424278 = queryNorm
              0.8276164 = fieldWeight in 2833, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.022745464 = weight(abstract_txt:purpose in 2833) [ClassicSimilarity], result of:
            0.022745464 = score(doc=2833,freq=2.0), product of:
              0.065626636 = queryWeight, product of:
                1.0908816 = boost
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.013424278 = queryNorm
              0.3465889 = fieldWeight in 2833, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.021093111 = weight(abstract_txt:questions in 2833) [ClassicSimilarity], result of:
            0.021093111 = score(doc=2833,freq=1.0), product of:
              0.07862984 = queryWeight, product of:
                1.1940739 = boost
                4.905296 = idf(docFreq=876, maxDocs=43556)
                0.013424278 = queryNorm
              0.26825836 = fieldWeight in 2833, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.905296 = idf(docFreq=876, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.03116482 = weight(abstract_txt:factors in 2833) [ClassicSimilarity], result of:
            0.03116482 = score(doc=2833,freq=2.0), product of:
              0.080958046 = queryWeight, product of:
                1.2116231 = boost
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.013424278 = queryNorm
              0.38495025 = fieldWeight in 2833, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.011363102 = weight(abstract_txt:paper in 2833) [ClassicSimilarity], result of:
            0.011363102 = score(doc=2833,freq=1.0), product of:
              0.059592094 = queryWeight, product of:
                1.2731439 = boost
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.013424278 = queryNorm
              0.19068137 = fieldWeight in 2833, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.065150775 = weight(abstract_txt:affect in 2833) [ClassicSimilarity], result of:
            0.065150775 = score(doc=2833,freq=3.0), product of:
              0.11562886 = queryWeight, product of:
                1.448007 = boost
                5.9484615 = idf(docFreq=308, maxDocs=43556)
                0.013424278 = queryNorm
              0.56344736 = fieldWeight in 2833, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.9484615 = idf(docFreq=308, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.0733992 = weight(abstract_txt:judgements in 2833) [ClassicSimilarity], result of:
            0.0733992 = score(doc=2833,freq=1.0), product of:
              0.18055984 = queryWeight, product of:
                1.8094563 = boost
                7.4333076 = idf(docFreq=69, maxDocs=43556)
                0.013424278 = queryNorm
              0.406509 = fieldWeight in 2833, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.4333076 = idf(docFreq=69, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.025103644 = weight(abstract_txt:research in 2833) [ClassicSimilarity], result of:
            0.025103644 = score(doc=2833,freq=3.0), product of:
              0.08309809 = queryWeight, product of:
                1.9408995 = boost
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.013424278 = queryNorm
              0.30209652 = fieldWeight in 2833, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.01889585 = weight(abstract_txt:that in 2833) [ClassicSimilarity], result of:
            0.01889585 = score(doc=2833,freq=5.0), product of:
              0.06487896 = queryWeight, product of:
                2.0291936 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.013424278 = queryNorm
              0.29124773 = fieldWeight in 2833, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.22310606 = weight(abstract_txt:relevance in 2833) [ClassicSimilarity], result of:
            0.22310606 = score(doc=2833,freq=3.0), product of:
              0.47735834 = queryWeight, product of:
                7.206683 = boost
                4.934216 = idf(docFreq=851, maxDocs=43556)
                0.013424278 = queryNorm
              0.46737647 = fieldWeight in 2833, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.934216 = idf(docFreq=851, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
          0.5354145 = weight(abstract_txt:trec in 2833) [ClassicSimilarity], result of:
            0.5354145 = score(doc=2833,freq=4.0), product of:
              0.7315719 = queryWeight, product of:
                8.144249 = boost
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.013424278 = queryNorm
              0.7318686 = fieldWeight in 2833, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.0546875 = fieldNorm(doc=2833)
        0.48 = coord(12/25)
    
  3. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.43
    0.4304033 = sum of:
      0.4304033 = product of:
        1.0760082 = sum of:
          0.014953824 = weight(abstract_txt:findings in 4589) [ClassicSimilarity], result of:
            0.014953824 = score(doc=4589,freq=1.0), product of:
              0.057191808 = queryWeight, product of:
                1.0183673 = boost
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.013424278 = queryNorm
              0.26146793 = fieldWeight in 4589, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.117980115 = weight(abstract_txt:assessors in 4589) [ClassicSimilarity], result of:
            0.117980115 = score(doc=4589,freq=3.0), product of:
              0.12473484 = queryWeight, product of:
                1.0634484 = boost
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.013424278 = queryNorm
              0.94584733 = fieldWeight in 4589, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.018381111 = weight(abstract_txt:purpose in 4589) [ClassicSimilarity], result of:
            0.018381111 = score(doc=4589,freq=1.0), product of:
              0.065626636 = queryWeight, product of:
                1.0908816 = boost
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.013424278 = queryNorm
              0.28008613 = fieldWeight in 4589, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.021265876 = weight(abstract_txt:collections in 4589) [ClassicSimilarity], result of:
            0.021265876 = score(doc=4589,freq=1.0), product of:
              0.072324894 = queryWeight, product of:
                1.1452003 = boost
                4.704521 = idf(docFreq=1071, maxDocs=43556)
                0.013424278 = queryNorm
              0.29403257 = fieldWeight in 4589, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.704521 = idf(docFreq=1071, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.025184978 = weight(abstract_txt:factors in 4589) [ClassicSimilarity], result of:
            0.025184978 = score(doc=4589,freq=1.0), product of:
              0.080958046 = queryWeight, product of:
                1.2116231 = boost
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.013424278 = queryNorm
              0.31108677 = fieldWeight in 4589, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.026362171 = weight(abstract_txt:test in 4589) [ClassicSimilarity], result of:
            0.026362171 = score(doc=4589,freq=1.0), product of:
              0.083461545 = queryWeight, product of:
                1.2302142 = boost
                5.0537615 = idf(docFreq=755, maxDocs=43556)
                0.013424278 = queryNorm
              0.3158601 = fieldWeight in 4589, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0537615 = idf(docFreq=755, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.012986403 = weight(abstract_txt:paper in 4589) [ClassicSimilarity], result of:
            0.012986403 = score(doc=4589,freq=1.0), product of:
              0.059592094 = queryWeight, product of:
                1.2731439 = boost
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.013424278 = queryNorm
              0.21792157 = fieldWeight in 4589, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.016727613 = weight(abstract_txt:that in 4589) [ClassicSimilarity], result of:
            0.016727613 = score(doc=4589,freq=3.0), product of:
              0.06487896 = queryWeight, product of:
                2.0291936 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.013424278 = queryNorm
              0.257828 = fieldWeight in 4589, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.38948587 = weight(abstract_txt:relevance in 4589) [ClassicSimilarity], result of:
            0.38948587 = score(doc=4589,freq=7.0), product of:
              0.47735834 = queryWeight, product of:
                7.206683 = boost
                4.934216 = idf(docFreq=851, maxDocs=43556)
                0.013424278 = queryNorm
              0.8159193 = fieldWeight in 4589, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.934216 = idf(docFreq=851, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
          0.43268025 = weight(abstract_txt:trec in 4589) [ClassicSimilarity], result of:
            0.43268025 = score(doc=4589,freq=2.0), product of:
              0.7315719 = queryWeight, product of:
                8.144249 = boost
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.013424278 = queryNorm
              0.5914391 = fieldWeight in 4589, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.0625 = fieldNorm(doc=4589)
        0.4 = coord(10/25)
    
  4. Sparck Jones, K.: Reflections on TREC (1997) 0.28
    0.27812758 = sum of:
      0.27812758 = product of:
        0.8691487 = sum of:
          0.022430737 = weight(abstract_txt:findings in 1578) [ClassicSimilarity], result of:
            0.022430737 = score(doc=1578,freq=1.0), product of:
              0.057191808 = queryWeight, product of:
                1.0183673 = boost
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.013424278 = queryNorm
              0.3922019 = fieldWeight in 1578, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.037777465 = weight(abstract_txt:factors in 1578) [ClassicSimilarity], result of:
            0.037777465 = score(doc=1578,freq=1.0), product of:
              0.080958046 = queryWeight, product of:
                1.2116231 = boost
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.013424278 = queryNorm
              0.46663016 = fieldWeight in 1578, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.9773884 = idf(docFreq=815, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.05592261 = weight(abstract_txt:test in 1578) [ClassicSimilarity], result of:
            0.05592261 = score(doc=1578,freq=2.0), product of:
              0.083461545 = queryWeight, product of:
                1.2302142 = boost
                5.0537615 = idf(docFreq=755, maxDocs=43556)
                0.013424278 = queryNorm
              0.6700404 = fieldWeight in 1578, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.0537615 = idf(docFreq=755, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.019479604 = weight(abstract_txt:paper in 1578) [ClassicSimilarity], result of:
            0.019479604 = score(doc=1578,freq=1.0), product of:
              0.059592094 = queryWeight, product of:
                1.2731439 = boost
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.013424278 = queryNorm
              0.32688236 = fieldWeight in 1578, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.03513778 = weight(abstract_txt:research in 1578) [ClassicSimilarity], result of:
            0.03513778 = score(doc=1578,freq=2.0), product of:
              0.08309809 = queryWeight, product of:
                1.9408995 = boost
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.013424278 = queryNorm
              0.42284703 = fieldWeight in 1578, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.02048706 = weight(abstract_txt:that in 1578) [ClassicSimilarity], result of:
            0.02048706 = score(doc=1578,freq=2.0), product of:
              0.06487896 = queryWeight, product of:
                2.0291936 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.013424278 = queryNorm
              0.31577355 = fieldWeight in 1578, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.028893042 = weight(abstract_txt:data in 1578) [ClassicSimilarity], result of:
            0.028893042 = score(doc=1578,freq=1.0), product of:
              0.091892555 = queryWeight, product of:
                2.041022 = boost
                3.3538349 = idf(docFreq=4137, maxDocs=43556)
                0.013424278 = queryNorm
              0.314422 = fieldWeight in 1578, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.3538349 = idf(docFreq=4137, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
          0.6490204 = weight(abstract_txt:trec in 1578) [ClassicSimilarity], result of:
            0.6490204 = score(doc=1578,freq=2.0), product of:
              0.7315719 = queryWeight, product of:
                8.144249 = boost
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.013424278 = queryNorm
              0.8871587 = fieldWeight in 1578, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.09375 = fieldNorm(doc=1578)
        0.32 = coord(8/25)
    
  5. Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017) 0.25
    0.24806291 = sum of:
      0.24806291 = product of:
        0.56377935 = sum of:
          0.014953824 = weight(abstract_txt:findings in 698) [ClassicSimilarity], result of:
            0.014953824 = score(doc=698,freq=1.0), product of:
              0.057191808 = queryWeight, product of:
                1.0183673 = boost
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.013424278 = queryNorm
              0.26146793 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.183487 = idf(docFreq=1804, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.06811585 = weight(abstract_txt:assessors in 698) [ClassicSimilarity], result of:
            0.06811585 = score(doc=698,freq=1.0), product of:
              0.12473484 = queryWeight, product of:
                1.0634484 = boost
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.013424278 = queryNorm
              0.54608524 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.737364 = idf(docFreq=18, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.02594298 = weight(abstract_txt:types in 698) [ClassicSimilarity], result of:
            0.02594298 = score(doc=698,freq=2.0), product of:
              0.06553936 = queryWeight, product of:
                1.090156 = boost
                4.4783974 = idf(docFreq=1343, maxDocs=43556)
                0.013424278 = queryNorm
              0.39583814 = fieldWeight in 698, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4783974 = idf(docFreq=1343, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.018381111 = weight(abstract_txt:purpose in 698) [ClassicSimilarity], result of:
            0.018381111 = score(doc=698,freq=1.0), product of:
              0.065626636 = queryWeight, product of:
                1.0908816 = boost
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.013424278 = queryNorm
              0.28008613 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.481378 = idf(docFreq=1339, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.026362171 = weight(abstract_txt:test in 698) [ClassicSimilarity], result of:
            0.026362171 = score(doc=698,freq=1.0), product of:
              0.083461545 = queryWeight, product of:
                1.2302142 = boost
                5.0537615 = idf(docFreq=755, maxDocs=43556)
                0.013424278 = queryNorm
              0.3158601 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0537615 = idf(docFreq=755, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.012986403 = weight(abstract_txt:paper in 698) [ClassicSimilarity], result of:
            0.012986403 = score(doc=698,freq=1.0), product of:
              0.059592094 = queryWeight, product of:
                1.2731439 = boost
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.013424278 = queryNorm
              0.21792157 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.486745 = idf(docFreq=3622, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.01656411 = weight(abstract_txt:research in 698) [ClassicSimilarity], result of:
            0.01656411 = score(doc=698,freq=1.0), product of:
              0.08309809 = queryWeight, product of:
                1.9408995 = boost
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.013424278 = queryNorm
              0.19933201 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.1893122 = idf(docFreq=4877, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.009657692 = weight(abstract_txt:that in 698) [ClassicSimilarity], result of:
            0.009657692 = score(doc=698,freq=1.0), product of:
              0.06487896 = queryWeight, product of:
                2.0291936 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.013424278 = queryNorm
              0.14885707 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.019262029 = weight(abstract_txt:data in 698) [ClassicSimilarity], result of:
            0.019262029 = score(doc=698,freq=1.0), product of:
              0.091892555 = queryWeight, product of:
                2.041022 = boost
                3.3538349 = idf(docFreq=4137, maxDocs=43556)
                0.013424278 = queryNorm
              0.20961468 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.3538349 = idf(docFreq=4137, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.045602042 = weight(abstract_txt:studies in 698) [ClassicSimilarity], result of:
            0.045602042 = score(doc=698,freq=2.0), product of:
              0.12026941 = queryWeight, product of:
                2.088479 = boost
                4.289771 = idf(docFreq=1622, maxDocs=43556)
                0.013424278 = queryNorm
              0.37916577 = fieldWeight in 698, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.289771 = idf(docFreq=1622, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
          0.30595115 = weight(abstract_txt:trec in 698) [ClassicSimilarity], result of:
            0.30595115 = score(doc=698,freq=1.0), product of:
              0.7315719 = queryWeight, product of:
                8.144249 = boost
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.013424278 = queryNorm
              0.41821063 = fieldWeight in 698, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.69137 = idf(docFreq=146, maxDocs=43556)
                0.0625 = fieldNorm(doc=698)
        0.44 = coord(11/25)