Document (#38787)

Author
Ruthven, I.
Title
Relevance behaviour in TREC
Source
Journal of documentation. 70(2014) no.6, S.1098-1117
Year
2014
Abstract
Purpose - The purpose of this paper is to examine how various types of TREC data can be used to better understand relevance and serve as test-bed for exploring relevance. The author proposes that there are many interesting studies that can be performed on the TREC data collections that are not directly related to evaluating systems but to learning more about human judgements of information and relevance and that these studies can provide useful research questions for other types of investigation. Design/methodology/approach - Through several case studies the author shows how existing data from TREC can be used to learn more about the factors that may affect relevance judgements and interactive search decisions and answer new research questions for exploring relevance. Findings - The paper uncovers factors, such as familiarity, interest and strictness of relevance criteria, that affect the nature of relevance assessments within TREC, contrasting these against findings from user studies of relevance. Research limitations/implications - The research only considers certain uses of TREC data and assessment given by professional relevance assessors but motivates further exploration of the TREC data so that the research community can further exploit the effort involved in the construction of TREC test collections. Originality/value - The paper presents an original viewpoint on relevance investigations and TREC itself by motivating TREC as a source of inspiration on understanding relevance rather than purely as a source of evaluation material.
Content
Beitrag in einem Special Issue: Festschrift in honour of Nigel Ford
Theme
Retrievalstudien
Object
TREC

Similar documents (author)

  1. Ruthven, R.: Interactive information retrieval (2008) 5.20
    5.2002997 = sum of:
      5.2002997 = weight(author_txt:ruthven in 2839) [ClassicSimilarity], result of:
        5.2002997 = fieldWeight in 2839, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.320479 = idf(docFreq=27, maxDocs=42306)
          0.625 = fieldNorm(doc=2839)
    
  2. Ruthven, I.: Integrating approaches to relevance (2005) 5.20
    5.2002997 = sum of:
      5.2002997 = weight(author_txt:ruthven in 1764) [ClassicSimilarity], result of:
        5.2002997 = fieldWeight in 1764, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.320479 = idf(docFreq=27, maxDocs=42306)
          0.625 = fieldNorm(doc=1764)
    
  3. Ruthven, I.: ¬The language of information need : differentiating conscious and formalized information needs (2019) 5.20
    5.2002997 = sum of:
      5.2002997 = weight(author_txt:ruthven in 1954) [ClassicSimilarity], result of:
        5.2002997 = fieldWeight in 1954, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.320479 = idf(docFreq=27, maxDocs=42306)
          0.625 = fieldNorm(doc=1954)
    
  4. Lalmas, M.; Ruthven, I.: ¬A model for structured document retrieval : empirical investigations (1997) 4.16
    4.1602397 = sum of:
      4.1602397 = weight(author_txt:ruthven in 1728) [ClassicSimilarity], result of:
        4.1602397 = fieldWeight in 1728, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.320479 = idf(docFreq=27, maxDocs=42306)
          0.5 = fieldNorm(doc=1728)
    
  5. Lalmas, M.; Ruthven, I.: Representing and retrieving structured documents using the Dempster-Shafer theory of evidence : modelling and evaluation (1998) 4.16
    4.1602397 = sum of:
      4.1602397 = weight(author_txt:ruthven in 2077) [ClassicSimilarity], result of:
        4.1602397 = fieldWeight in 2077, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          8.320479 = idf(docFreq=27, maxDocs=42306)
          0.5 = fieldNorm(doc=2077)
    

Similar documents (content)

  1. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.60
    0.60377306 = sum of:
      0.60377306 = product of:
        1.3722115 = sum of:
          0.015802916 = weight(abstract_txt:findings in 1541) [ClassicSimilarity], result of:
            0.015802916 = score(doc=1541,freq=1.0), product of:
              0.059336 = queryWeight, product of:
                1.0286227 = boost
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.013537023 = queryNorm
              0.26632932 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.11680002 = weight(abstract_txt:assessors in 1541) [ClassicSimilarity], result of:
            0.11680002 = score(doc=1541,freq=3.0), product of:
              0.123900086 = queryWeight, product of:
                1.0510365 = boost
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.013537023 = queryNorm
              0.9426952 = fieldWeight in 1541, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.01904394 = weight(abstract_txt:purpose in 1541) [ClassicSimilarity], result of:
            0.01904394 = score(doc=1541,freq=1.0), product of:
              0.06719412 = queryWeight, product of:
                1.0946181 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.013537023 = queryNorm
              0.28341675 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.05145098 = weight(abstract_txt:factors in 1541) [ClassicSimilarity], result of:
            0.05145098 = score(doc=1541,freq=4.0), product of:
              0.082111225 = queryWeight, product of:
                1.2100359 = boost
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.013537023 = queryNorm
              0.62660104 = fieldWeight in 1541, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.044195637 = weight(abstract_txt:affect in 1541) [ClassicSimilarity], result of:
            0.044195637 = score(doc=1541,freq=1.0), product of:
              0.117782205 = queryWeight, product of:
                1.4492284 = boost
                6.00371 = idf(docFreq=283, maxDocs=42306)
                0.013537023 = queryNorm
              0.37523186 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.00371 = idf(docFreq=283, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.038402908 = weight(abstract_txt:research in 1541) [ClassicSimilarity], result of:
            0.038402908 = score(doc=1541,freq=5.0), product of:
              0.08512546 = queryWeight, product of:
                1.9480349 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.013537023 = queryNorm
              0.451133 = fieldWeight in 1541, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.01721932 = weight(abstract_txt:that in 1541) [ClassicSimilarity], result of:
            0.01721932 = score(doc=1541,freq=3.0), product of:
              0.06614338 = queryWeight, product of:
                2.0317683 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.013537023 = queryNorm
              0.26033324 = fieldWeight in 1541, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.019762397 = weight(abstract_txt:data in 1541) [ClassicSimilarity], result of:
            0.019762397 = score(doc=1541,freq=1.0), product of:
              0.09347594 = queryWeight, product of:
                2.0413473 = boost
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.013537023 = queryNorm
              0.21141694 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.03305808 = weight(abstract_txt:studies in 1541) [ClassicSimilarity], result of:
            0.03305808 = score(doc=1541,freq=1.0), product of:
              0.12227993 = queryWeight, product of:
                2.088284 = boost
                4.325561 = idf(docFreq=1520, maxDocs=42306)
                0.013537023 = queryNorm
              0.27034757 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.325561 = idf(docFreq=1520, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.49022752 = weight(abstract_txt:relevance in 1541) [ClassicSimilarity], result of:
            0.49022752 = score(doc=1541,freq=11.0), product of:
              0.47864267 = queryWeight, product of:
                7.1561313 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.013537023 = queryNorm
              1.0242035 = fieldWeight in 1541, product of:
                3.3166249 = tf(freq=11.0), with freq of:
                  11.0 = termFreq=11.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
          0.5262478 = weight(abstract_txt:trec in 1541) [ClassicSimilarity], result of:
            0.5262478 = score(doc=1541,freq=3.0), product of:
              0.7281763 = queryWeight, product of:
                8.057499 = boost
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.013537023 = queryNorm
              0.72269285 = fieldWeight in 1541, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.0625 = fieldNorm(doc=1541)
        0.44 = coord(11/25)
    
  2. Ruthven, I.; Baillie, M.; Elsweiler, D.: ¬The relative effects of knowledge, interest and confidence in assessing relevance (2007) 0.55
    0.54972196 = sum of:
      0.54972196 = product of:
        1.1452541 = sum of:
          0.013827552 = weight(abstract_txt:findings in 2836) [ClassicSimilarity], result of:
            0.013827552 = score(doc=2836,freq=1.0), product of:
              0.059336 = queryWeight, product of:
                1.0286227 = boost
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.013537023 = queryNorm
              0.23303816 = fieldWeight in 2836, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.10220002 = weight(abstract_txt:assessors in 2836) [ClassicSimilarity], result of:
            0.10220002 = score(doc=2836,freq=3.0), product of:
              0.123900086 = queryWeight, product of:
                1.0510365 = boost
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.013537023 = queryNorm
              0.8248583 = fieldWeight in 2836, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.023565672 = weight(abstract_txt:purpose in 2836) [ClassicSimilarity], result of:
            0.023565672 = score(doc=2836,freq=2.0), product of:
              0.06719412 = queryWeight, product of:
                1.0946181 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.013537023 = queryNorm
              0.35071033 = fieldWeight in 2836, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.021507854 = weight(abstract_txt:questions in 2836) [ClassicSimilarity], result of:
            0.021507854 = score(doc=2836,freq=1.0), product of:
              0.079656176 = queryWeight, product of:
                1.1918092 = boost
                4.9373007 = idf(docFreq=824, maxDocs=42306)
                0.013537023 = queryNorm
              0.27000862 = fieldWeight in 2836, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.9373007 = idf(docFreq=824, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.03183367 = weight(abstract_txt:factors in 2836) [ClassicSimilarity], result of:
            0.03183367 = score(doc=2836,freq=2.0), product of:
              0.082111225 = queryWeight, product of:
                1.2100359 = boost
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.013537023 = queryNorm
              0.38768962 = fieldWeight in 2836, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.011609238 = weight(abstract_txt:paper in 2836) [ClassicSimilarity], result of:
            0.011609238 = score(doc=2836,freq=1.0), product of:
              0.060448788 = queryWeight, product of:
                1.2715588 = boost
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.013537023 = queryNorm
              0.19205078 = fieldWeight in 2836, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.06698045 = weight(abstract_txt:affect in 2836) [ClassicSimilarity], result of:
            0.06698045 = score(doc=2836,freq=3.0), product of:
              0.117782205 = queryWeight, product of:
                1.4492284 = boost
                6.00371 = idf(docFreq=283, maxDocs=42306)
                0.013537023 = queryNorm
              0.5686806 = fieldWeight in 2836, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.00371 = idf(docFreq=283, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.07253721 = weight(abstract_txt:judgements in 2836) [ClassicSimilarity], result of:
            0.07253721 = score(doc=2836,freq=1.0), product of:
              0.17914113 = queryWeight, product of:
                1.7872884 = boost
                7.404189 = idf(docFreq=69, maxDocs=42306)
                0.013537023 = queryNorm
              0.40491658 = fieldWeight in 2836, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.404189 = idf(docFreq=69, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.026028417 = weight(abstract_txt:research in 2836) [ClassicSimilarity], result of:
            0.026028417 = score(doc=2836,freq=3.0), product of:
              0.08512546 = queryWeight, product of:
                1.9480349 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.013537023 = queryNorm
              0.30576536 = fieldWeight in 2836, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.019451292 = weight(abstract_txt:that in 2836) [ClassicSimilarity], result of:
            0.019451292 = score(doc=2836,freq=5.0), product of:
              0.06614338 = queryWeight, product of:
                2.0317683 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.013537023 = queryNorm
              0.2940777 = fieldWeight in 2836, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.22401135 = weight(abstract_txt:relevance in 2836) [ClassicSimilarity], result of:
            0.22401135 = score(doc=2836,freq=3.0), product of:
              0.47864267 = queryWeight, product of:
                7.1561313 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.013537023 = queryNorm
              0.46801373 = fieldWeight in 2836, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
          0.5317013 = weight(abstract_txt:trec in 2836) [ClassicSimilarity], result of:
            0.5317013 = score(doc=2836,freq=4.0), product of:
              0.7281763 = queryWeight, product of:
                8.057499 = boost
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.013537023 = queryNorm
              0.7301821 = fieldWeight in 2836, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.0546875 = fieldNorm(doc=2836)
        0.48 = coord(12/25)
    
  3. Ravana, S.D.; Rajagopal, P.; Balakrishnan, V.: Ranking retrieval systems using pseudo relevance judgments (2015) 0.43
    0.43058768 = sum of:
      0.43058768 = product of:
        1.0764692 = sum of:
          0.015802916 = weight(abstract_txt:findings in 4592) [ClassicSimilarity], result of:
            0.015802916 = score(doc=4592,freq=1.0), product of:
              0.059336 = queryWeight, product of:
                1.0286227 = boost
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.013537023 = queryNorm
              0.26632932 = fieldWeight in 4592, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.11680002 = weight(abstract_txt:assessors in 4592) [ClassicSimilarity], result of:
            0.11680002 = score(doc=4592,freq=3.0), product of:
              0.123900086 = queryWeight, product of:
                1.0510365 = boost
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.013537023 = queryNorm
              0.9426952 = fieldWeight in 4592, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.01904394 = weight(abstract_txt:purpose in 4592) [ClassicSimilarity], result of:
            0.01904394 = score(doc=4592,freq=1.0), product of:
              0.06719412 = queryWeight, product of:
                1.0946181 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.013537023 = queryNorm
              0.28341675 = fieldWeight in 4592, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.02128116 = weight(abstract_txt:collections in 4592) [ClassicSimilarity], result of:
            0.02128116 = score(doc=4592,freq=1.0), product of:
              0.07235862 = queryWeight, product of:
                1.1359053 = boost
                4.705708 = idf(docFreq=1039, maxDocs=42306)
                0.013537023 = queryNorm
              0.29410675 = fieldWeight in 4592, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.705708 = idf(docFreq=1039, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.02572549 = weight(abstract_txt:factors in 4592) [ClassicSimilarity], result of:
            0.02572549 = score(doc=4592,freq=1.0), product of:
              0.082111225 = queryWeight, product of:
                1.2100359 = boost
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.013537023 = queryNorm
              0.31330052 = fieldWeight in 4592, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.026582917 = weight(abstract_txt:test in 4592) [ClassicSimilarity], result of:
            0.026582917 = score(doc=4592,freq=1.0), product of:
              0.08392574 = queryWeight, product of:
                1.2233326 = boost
                5.067893 = idf(docFreq=723, maxDocs=42306)
                0.013537023 = queryNorm
              0.3167433 = fieldWeight in 4592, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.067893 = idf(docFreq=723, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.0132677 = weight(abstract_txt:paper in 4592) [ClassicSimilarity], result of:
            0.0132677 = score(doc=4592,freq=1.0), product of:
              0.060448788 = queryWeight, product of:
                1.2715588 = boost
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.013537023 = queryNorm
              0.21948661 = fieldWeight in 4592, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.01721932 = weight(abstract_txt:that in 4592) [ClassicSimilarity], result of:
            0.01721932 = score(doc=4592,freq=3.0), product of:
              0.06614338 = queryWeight, product of:
                2.0317683 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.013537023 = queryNorm
              0.26033324 = fieldWeight in 4592, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.39106625 = weight(abstract_txt:relevance in 4592) [ClassicSimilarity], result of:
            0.39106625 = score(doc=4592,freq=7.0), product of:
              0.47864267 = queryWeight, product of:
                7.1561313 = boost
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.013537023 = queryNorm
              0.81703174 = fieldWeight in 4592, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.9409437 = idf(docFreq=821, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
          0.4296795 = weight(abstract_txt:trec in 4592) [ClassicSimilarity], result of:
            0.4296795 = score(doc=4592,freq=2.0), product of:
              0.7281763 = queryWeight, product of:
                8.057499 = boost
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.013537023 = queryNorm
              0.5900762 = fieldWeight in 4592, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.0625 = fieldNorm(doc=4592)
        0.4 = coord(10/25)
    
  4. Sparck Jones, K.: Reflections on TREC (1997) 0.28
    0.2784862 = sum of:
      0.2784862 = product of:
        0.87026936 = sum of:
          0.023704376 = weight(abstract_txt:findings in 1581) [ClassicSimilarity], result of:
            0.023704376 = score(doc=1581,freq=1.0), product of:
              0.059336 = queryWeight, product of:
                1.0286227 = boost
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.013537023 = queryNorm
              0.399494 = fieldWeight in 1581, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.038588237 = weight(abstract_txt:factors in 1581) [ClassicSimilarity], result of:
            0.038588237 = score(doc=1581,freq=1.0), product of:
              0.082111225 = queryWeight, product of:
                1.2100359 = boost
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.013537023 = queryNorm
              0.4699508 = fieldWeight in 1581, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0128083 = idf(docFreq=764, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.056390878 = weight(abstract_txt:test in 1581) [ClassicSimilarity], result of:
            0.056390878 = score(doc=1581,freq=2.0), product of:
              0.08392574 = queryWeight, product of:
                1.2233326 = boost
                5.067893 = idf(docFreq=723, maxDocs=42306)
                0.013537023 = queryNorm
              0.671914 = fieldWeight in 1581, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.067893 = idf(docFreq=723, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.01990155 = weight(abstract_txt:paper in 1581) [ClassicSimilarity], result of:
            0.01990155 = score(doc=1581,freq=1.0), product of:
              0.060448788 = queryWeight, product of:
                1.2715588 = boost
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.013537023 = queryNorm
              0.32922992 = fieldWeight in 1581, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.03643219 = weight(abstract_txt:research in 1581) [ClassicSimilarity], result of:
            0.03643219 = score(doc=1581,freq=2.0), product of:
              0.08512546 = queryWeight, product of:
                1.9480349 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.013537023 = queryNorm
              0.42798233 = fieldWeight in 1581, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.021089274 = weight(abstract_txt:that in 1581) [ClassicSimilarity], result of:
            0.021089274 = score(doc=1581,freq=2.0), product of:
              0.06614338 = queryWeight, product of:
                2.0317683 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.013537023 = queryNorm
              0.31884181 = fieldWeight in 1581, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.029643595 = weight(abstract_txt:data in 1581) [ClassicSimilarity], result of:
            0.029643595 = score(doc=1581,freq=1.0), product of:
              0.09347594 = queryWeight, product of:
                2.0413473 = boost
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.013537023 = queryNorm
              0.3171254 = fieldWeight in 1581, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
          0.64451927 = weight(abstract_txt:trec in 1581) [ClassicSimilarity], result of:
            0.64451927 = score(doc=1581,freq=2.0), product of:
              0.7281763 = queryWeight, product of:
                8.057499 = boost
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.013537023 = queryNorm
              0.8851143 = fieldWeight in 1581, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.09375 = fieldNorm(doc=1581)
        0.32 = coord(8/25)
    
  5. Behnert, C.; Lewandowski, D.: ¬A framework for designing retrieval effectiveness studies of library information systems using human relevance assessments (2017) 0.25
    0.2490157 = sum of:
      0.2490157 = product of:
        0.5659448 = sum of:
          0.015802916 = weight(abstract_txt:findings in 619) [ClassicSimilarity], result of:
            0.015802916 = score(doc=619,freq=1.0), product of:
              0.059336 = queryWeight, product of:
                1.0286227 = boost
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.013537023 = queryNorm
              0.26632932 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.261269 = idf(docFreq=1621, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.06743452 = weight(abstract_txt:assessors in 619) [ClassicSimilarity], result of:
            0.06743452 = score(doc=619,freq=1.0), product of:
              0.123900086 = queryWeight, product of:
                1.0510365 = boost
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.013537023 = queryNorm
              0.54426533 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.708245 = idf(docFreq=18, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.026354015 = weight(abstract_txt:types in 619) [ClassicSimilarity], result of:
            0.026354015 = score(doc=619,freq=2.0), product of:
              0.06622896 = queryWeight, product of:
                1.0867282 = boost
                4.5019827 = idf(docFreq=1274, maxDocs=42306)
                0.013537023 = queryNorm
              0.3979228 = fieldWeight in 619, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.5019827 = idf(docFreq=1274, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.01904394 = weight(abstract_txt:purpose in 619) [ClassicSimilarity], result of:
            0.01904394 = score(doc=619,freq=1.0), product of:
              0.06719412 = queryWeight, product of:
                1.0946181 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.013537023 = queryNorm
              0.28341675 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.026582917 = weight(abstract_txt:test in 619) [ClassicSimilarity], result of:
            0.026582917 = score(doc=619,freq=1.0), product of:
              0.08392574 = queryWeight, product of:
                1.2233326 = boost
                5.067893 = idf(docFreq=723, maxDocs=42306)
                0.013537023 = queryNorm
              0.3167433 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.067893 = idf(docFreq=723, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.0132677 = weight(abstract_txt:paper in 619) [ClassicSimilarity], result of:
            0.0132677 = score(doc=619,freq=1.0), product of:
              0.060448788 = queryWeight, product of:
                1.2715588 = boost
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.013537023 = queryNorm
              0.21948661 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.5117857 = idf(docFreq=3431, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.017174302 = weight(abstract_txt:research in 619) [ClassicSimilarity], result of:
            0.017174302 = score(doc=619,freq=1.0), product of:
              0.08512546 = queryWeight, product of:
                1.9480349 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.013537023 = queryNorm
              0.20175281 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.009941579 = weight(abstract_txt:that in 619) [ClassicSimilarity], result of:
            0.009941579 = score(doc=619,freq=1.0), product of:
              0.06614338 = queryWeight, product of:
                2.0317683 = boost
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.013537023 = queryNorm
              0.15030347 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.4048555 = idf(docFreq=10381, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.019762397 = weight(abstract_txt:data in 619) [ClassicSimilarity], result of:
            0.019762397 = score(doc=619,freq=1.0), product of:
              0.09347594 = queryWeight, product of:
                2.0413473 = boost
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.013537023 = queryNorm
              0.21141694 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.046751186 = weight(abstract_txt:studies in 619) [ClassicSimilarity], result of:
            0.046751186 = score(doc=619,freq=2.0), product of:
              0.12227993 = queryWeight, product of:
                2.088284 = boost
                4.325561 = idf(docFreq=1520, maxDocs=42306)
                0.013537023 = queryNorm
              0.3823292 = fieldWeight in 619, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.325561 = idf(docFreq=1520, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
          0.3038293 = weight(abstract_txt:trec in 619) [ClassicSimilarity], result of:
            0.3038293 = score(doc=619,freq=1.0), product of:
              0.7281763 = queryWeight, product of:
                8.057499 = boost
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.013537023 = queryNorm
              0.4172469 = fieldWeight in 619, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.6759505 = idf(docFreq=144, maxDocs=42306)
                0.0625 = fieldNorm(doc=619)
        0.44 = coord(11/25)