Document (#30215)

Author
Tang, R.
Solomon, P.
Title
Use of relevance criteria across stages of document evaluation : on the complementarity of experimental and naturalistic studies
Source
Journal of the American Society for Information Science and technology. 52(2001) no.8, S.676-687
Year
2001
Abstract
Tang and Solomon, based upon their review of the history of topical and other than topical criteria in relevance evaluation, decide to look at a two stage model where judgements are first made on surrogate records and then on full document text to determine if a criteria shift takes place and if so in what manner and to what degree. Both a controlled experiment and a naturalistic study were used to study the staging of relevance judgement criteria. In the controlled environment 90 undergraduate Psychology students were instructed to choose papers that would help them meet an assignment from 20 preselected papers on broader topic that included that assigned. They first selected on the basis of citation and abstract, then read the papers, and in each process filled out a questionnaire on the importance of each of 15 criteria at each stage of the two-stage process. In the naturalistic study 9 Ph.D. Psychology students conducted literature searches to support their own research and were asked to think aloud while making their decisions from retrieved surrogates, and later filled out a questionnaire while reading those materials that they selected and then interviewed at the end of the process. Apparently understandability is important at both stages. Importance increased at stage two. Cognitive criteria do not all follow the same pattern across stages. The controlled group thought quality of information was most important in stage one and topicality most important in stage 2. In the naturalistic study topicality was most frequent for stage one and research structure for stage two. A classification of criteria by their functionality is suggested as a better approach. First a division as to whether a criterion is objectively associated with the document as opposed to being subjectively associated with a person's expectations; then a division based on primary (essential) or secondary (for assistance) status.
Theme
Benutzerstudien

Similar documents (author)

  1. Tang, R.; Solomon, P.: Toward an understanding of the dynamics of relevance judgement : an analysis of one person's search behavior (1998) 5.94
    5.9392343 = sum of:
      5.9392343 = sum of:
        2.67371 = weight(author_txt:tang in 5269) [ClassicSimilarity], result of:
          2.67371 = score(doc=5269,freq=1.0), product of:
            0.6585901 = queryWeight, product of:
              8.119497 = idf(docFreq=34, maxDocs=43254)
              0.081112176 = queryNorm
            4.0597486 = fieldWeight in 5269, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.119497 = idf(docFreq=34, maxDocs=43254)
              0.5 = fieldNorm(doc=5269)
        3.2655241 = weight(author_txt:solomon in 5269) [ClassicSimilarity], result of:
          3.2655241 = score(doc=5269,freq=1.0), product of:
            0.75250185 = queryWeight, product of:
              1.0689224 = boost
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.081112176 = queryNorm
            4.339556 = fieldWeight in 5269, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.679112 = idf(docFreq=19, maxDocs=43254)
              0.5 = fieldNorm(doc=5269)
    
  2. Solomon, P.: Children's information retrieval behavior : a case analysis of an OPAC (1993) 2.04
    2.0409524 = sum of:
      2.0409524 = product of:
        4.081905 = sum of:
          4.081905 = weight(author_txt:solomon in 4349) [ClassicSimilarity], result of:
            4.081905 = score(doc=4349,freq=1.0), product of:
              0.75250185 = queryWeight, product of:
                1.0689224 = boost
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.081112176 = queryNorm
              5.424445 = fieldWeight in 4349, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.625 = fieldNorm(doc=4349)
        0.5 = coord(1/2)
    
  3. Solomon, P.: On the dynamics of information system use : from novice to? (1992) 2.04
    2.0409524 = sum of:
      2.0409524 = product of:
        4.081905 = sum of:
          4.081905 = weight(author_txt:solomon in 4519) [ClassicSimilarity], result of:
            4.081905 = score(doc=4519,freq=1.0), product of:
              0.75250185 = queryWeight, product of:
                1.0689224 = boost
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.081112176 = queryNorm
              5.424445 = fieldWeight in 4519, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.625 = fieldNorm(doc=4519)
        0.5 = coord(1/2)
    
  4. Solomon, P.: Children, technology, and instruction : a case study of elementary school children using an online public access catalog (OPAC) (1994) 2.04
    2.0409524 = sum of:
      2.0409524 = product of:
        4.081905 = sum of:
          4.081905 = weight(author_txt:solomon in 2492) [ClassicSimilarity], result of:
            4.081905 = score(doc=2492,freq=1.0), product of:
              0.75250185 = queryWeight, product of:
                1.0689224 = boost
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.081112176 = queryNorm
              5.424445 = fieldWeight in 2492, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.625 = fieldNorm(doc=2492)
        0.5 = coord(1/2)
    
  5. Solomon, P.: Dicovering information bahavior in sense making : I. time and timing (1997) 2.04
    2.0409524 = sum of:
      2.0409524 = product of:
        4.081905 = sum of:
          4.081905 = weight(author_txt:solomon in 1958) [ClassicSimilarity], result of:
            4.081905 = score(doc=1958,freq=1.0), product of:
              0.75250185 = queryWeight, product of:
                1.0689224 = boost
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.081112176 = queryNorm
              5.424445 = fieldWeight in 1958, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.679112 = idf(docFreq=19, maxDocs=43254)
                0.625 = fieldNorm(doc=1958)
        0.5 = coord(1/2)
    

Similar documents (content)

  1. Wang, P.; White, M.D.: ¬A cognitive model of document use during a research project : Study II: Decisions at the reading and citing stages (1999) 0.35
    0.3549374 = sum of:
      0.3549374 = product of:
        0.8066759 = sum of:
          0.030641103 = weight(abstract_txt:selected in 4941) [ClassicSimilarity], result of:
            0.030641103 = score(doc=4941,freq=1.0), product of:
              0.093583934 = queryWeight, product of:
                1.0201422 = boost
                5.2386947 = idf(docFreq=623, maxDocs=43254)
                0.017511263 = queryNorm
              0.32741842 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.2386947 = idf(docFreq=623, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.015969869 = weight(abstract_txt:were in 4941) [ClassicSimilarity], result of:
            0.015969869 = score(doc=4941,freq=1.0), product of:
              0.06937926 = queryWeight, product of:
                1.0757728 = boost
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.017511263 = queryNorm
              0.23018216 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.030345894 = weight(abstract_txt:process in 4941) [ClassicSimilarity], result of:
            0.030345894 = score(doc=4941,freq=2.0), product of:
              0.084479645 = queryWeight, product of:
                1.1870848 = boost
                4.063992 = idf(docFreq=2019, maxDocs=43254)
                0.017511263 = queryNorm
              0.35920954 = fieldWeight in 4941, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.063992 = idf(docFreq=2019, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.022442581 = weight(abstract_txt:each in 4941) [ClassicSimilarity], result of:
            0.022442581 = score(doc=4941,freq=1.0), product of:
              0.08704503 = queryWeight, product of:
                1.204974 = boost
                4.125236 = idf(docFreq=1899, maxDocs=43254)
                0.017511263 = queryNorm
              0.25782725 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.125236 = idf(docFreq=1899, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.023467667 = weight(abstract_txt:first in 4941) [ClassicSimilarity], result of:
            0.023467667 = score(doc=4941,freq=1.0), product of:
              0.08967583 = queryWeight, product of:
                1.2230477 = boost
                4.1871114 = idf(docFreq=1785, maxDocs=43254)
                0.017511263 = queryNorm
              0.26169446 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1871114 = idf(docFreq=1785, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.013729636 = weight(abstract_txt:their in 4941) [ClassicSimilarity], result of:
            0.013729636 = score(doc=4941,freq=1.0), product of:
              0.06904216 = queryWeight, product of:
                1.2391739 = boost
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.017511263 = queryNorm
              0.19885872 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.05620488 = weight(abstract_txt:document in 4941) [ClassicSimilarity], result of:
            0.05620488 = score(doc=4941,freq=5.0), product of:
              0.09387605 = queryWeight, product of:
                1.2513624 = boost
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.017511263 = queryNorm
              0.59871376 = fieldWeight in 4941, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.04640197 = weight(abstract_txt:study in 4941) [ClassicSimilarity], result of:
            0.04640197 = score(doc=4941,freq=7.0), product of:
              0.08128305 = queryWeight, product of:
                1.3445442 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.017511263 = queryNorm
              0.57086897 = fieldWeight in 4941, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.08118097 = weight(abstract_txt:stages in 4941) [ClassicSimilarity], result of:
            0.08118097 = score(doc=4941,freq=1.0), product of:
              0.2051154 = queryWeight, product of:
                1.849715 = boost
                6.332511 = idf(docFreq=208, maxDocs=43254)
                0.017511263 = queryNorm
              0.39578193 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.332511 = idf(docFreq=208, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.29069954 = weight(abstract_txt:criteria in 4941) [ClassicSimilarity], result of:
            0.29069954 = score(doc=4941,freq=5.0), product of:
              0.37238652 = queryWeight, product of:
                3.807072 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.017511263 = queryNorm
              0.7806393 = fieldWeight in 4941, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
          0.19559182 = weight(abstract_txt:stage in 4941) [ClassicSimilarity], result of:
            0.19559182 = score(doc=4941,freq=1.0), product of:
              0.5111937 = queryWeight, product of:
                4.7685113 = boost
                6.1218853 = idf(docFreq=257, maxDocs=43254)
                0.017511263 = queryNorm
              0.38261783 = fieldWeight in 4941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.1218853 = idf(docFreq=257, maxDocs=43254)
                0.0625 = fieldNorm(doc=4941)
        0.44 = coord(11/25)
    
  2. Choi, Y.; Rasmussen, E.M.: Users' relevance criteria in image retrieval in American history (2002) 0.34
    0.33812687 = sum of:
      0.33812687 = product of:
        0.76847017 = sum of:
          0.025254007 = weight(abstract_txt:across in 4593) [ClassicSimilarity], result of:
            0.025254007 = score(doc=4593,freq=1.0), product of:
              0.08992488 = queryWeight, product of:
                5.135259 = idf(docFreq=691, maxDocs=43254)
                0.017511263 = queryNorm
              0.2808345 = fieldWeight in 4593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.135259 = idf(docFreq=691, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.019761704 = weight(abstract_txt:were in 4593) [ClassicSimilarity], result of:
            0.019761704 = score(doc=4593,freq=2.0), product of:
              0.06937926 = queryWeight, product of:
                1.0757728 = boost
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.017511263 = queryNorm
              0.28483588 = fieldWeight in 4593, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.017296998 = weight(abstract_txt:most in 4593) [ClassicSimilarity], result of:
            0.017296998 = score(doc=4593,freq=1.0), product of:
              0.07998414 = queryWeight, product of:
                1.1550683 = boost
                3.9543834 = idf(docFreq=2253, maxDocs=43254)
                0.017511263 = queryNorm
              0.21625534 = fieldWeight in 4593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.9543834 = idf(docFreq=2253, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.019637257 = weight(abstract_txt:each in 4593) [ClassicSimilarity], result of:
            0.019637257 = score(doc=4593,freq=1.0), product of:
              0.08704503 = queryWeight, product of:
                1.204974 = boost
                4.125236 = idf(docFreq=1899, maxDocs=43254)
                0.017511263 = queryNorm
              0.22559884 = fieldWeight in 4593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.125236 = idf(docFreq=1899, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.02997882 = weight(abstract_txt:important in 4593) [ClassicSimilarity], result of:
            0.02997882 = score(doc=4593,freq=2.0), product of:
              0.0915988 = queryWeight, product of:
                1.2360915 = boost
                4.2317667 = idf(docFreq=1707, maxDocs=43254)
                0.017511263 = queryNorm
              0.32728398 = fieldWeight in 4593, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.2317667 = idf(docFreq=1707, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.012013432 = weight(abstract_txt:their in 4593) [ClassicSimilarity], result of:
            0.012013432 = score(doc=4593,freq=1.0), product of:
              0.06904216 = queryWeight, product of:
                1.2391739 = boost
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.017511263 = queryNorm
              0.17400138 = fieldWeight in 4593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.03431472 = weight(abstract_txt:study in 4593) [ClassicSimilarity], result of:
            0.03431472 = score(doc=4593,freq=5.0), product of:
              0.08128305 = queryWeight, product of:
                1.3445442 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.017511263 = queryNorm
              0.42216334 = fieldWeight in 4593, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.08884947 = weight(abstract_txt:relevance in 4593) [ClassicSimilarity], result of:
            0.08884947 = score(doc=4593,freq=7.0), product of:
              0.12447839 = queryWeight, product of:
                1.4409626 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.017511263 = queryNorm
              0.71377426 = fieldWeight in 4593, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.14936511 = weight(abstract_txt:topicality in 4593) [ClassicSimilarity], result of:
            0.14936511 = score(doc=4593,freq=2.0), product of:
              0.23342595 = queryWeight, product of:
                1.611145 = boost
                8.273647 = idf(docFreq=29, maxDocs=43254)
                0.017511263 = queryNorm
              0.6398822 = fieldWeight in 4593, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                8.273647 = idf(docFreq=29, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.07103334 = weight(abstract_txt:stages in 4593) [ClassicSimilarity], result of:
            0.07103334 = score(doc=4593,freq=1.0), product of:
              0.2051154 = queryWeight, product of:
                1.849715 = boost
                6.332511 = idf(docFreq=208, maxDocs=43254)
                0.017511263 = queryNorm
              0.34630919 = fieldWeight in 4593, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.332511 = idf(docFreq=208, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
          0.30096528 = weight(abstract_txt:criteria in 4593) [ClassicSimilarity], result of:
            0.30096528 = score(doc=4593,freq=7.0), product of:
              0.37238652 = queryWeight, product of:
                3.807072 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.017511263 = queryNorm
              0.80820674 = fieldWeight in 4593, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0546875 = fieldNorm(doc=4593)
        0.44 = coord(11/25)
    
  3. Taylor, A.; Zhang, X.; Amadio, W.J.: Examination of relevance criteria choices and the information search process (2009) 0.30
    0.30337968 = sum of:
      0.30337968 = product of:
        0.9480615 = sum of:
          0.031939737 = weight(abstract_txt:were in 73) [ClassicSimilarity], result of:
            0.031939737 = score(doc=73,freq=4.0), product of:
              0.06937926 = queryWeight, product of:
                1.0757728 = boost
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.017511263 = queryNorm
              0.4603643 = fieldWeight in 73, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.047981072 = weight(abstract_txt:process in 73) [ClassicSimilarity], result of:
            0.047981072 = score(doc=73,freq=5.0), product of:
              0.084479645 = queryWeight, product of:
                1.1870848 = boost
                4.063992 = idf(docFreq=2019, maxDocs=43254)
                0.017511263 = queryNorm
              0.56796014 = fieldWeight in 73, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                4.063992 = idf(docFreq=2019, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.03037722 = weight(abstract_txt:study in 73) [ClassicSimilarity], result of:
            0.03037722 = score(doc=73,freq=3.0), product of:
              0.08128305 = queryWeight, product of:
                1.3445442 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.017511263 = queryNorm
              0.37372148 = fieldWeight in 73, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.10154226 = weight(abstract_txt:relevance in 73) [ClassicSimilarity], result of:
            0.10154226 = score(doc=73,freq=7.0), product of:
              0.12447839 = queryWeight, product of:
                1.4409626 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.017511263 = queryNorm
              0.815742 = fieldWeight in 73, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.0522151 = weight(abstract_txt:controlled in 73) [ClassicSimilarity], result of:
            0.0522151 = score(doc=73,freq=1.0), product of:
              0.15283619 = queryWeight, product of:
                1.5966834 = boost
                5.466255 = idf(docFreq=496, maxDocs=43254)
                0.017511263 = queryNorm
              0.34164095 = fieldWeight in 73, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.466255 = idf(docFreq=496, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.12070525 = weight(abstract_txt:topicality in 73) [ClassicSimilarity], result of:
            0.12070525 = score(doc=73,freq=1.0), product of:
              0.23342595 = queryWeight, product of:
                1.611145 = boost
                8.273647 = idf(docFreq=29, maxDocs=43254)
                0.017511263 = queryNorm
              0.51710296 = fieldWeight in 73, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.273647 = idf(docFreq=29, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.36770907 = weight(abstract_txt:criteria in 73) [ClassicSimilarity], result of:
            0.36770907 = score(doc=73,freq=8.0), product of:
              0.37238652 = queryWeight, product of:
                3.807072 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.017511263 = queryNorm
              0.9874393 = fieldWeight in 73, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
          0.19559182 = weight(abstract_txt:stage in 73) [ClassicSimilarity], result of:
            0.19559182 = score(doc=73,freq=1.0), product of:
              0.5111937 = queryWeight, product of:
                4.7685113 = boost
                6.1218853 = idf(docFreq=257, maxDocs=43254)
                0.017511263 = queryNorm
              0.38261783 = fieldWeight in 73, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.1218853 = idf(docFreq=257, maxDocs=43254)
                0.0625 = fieldNorm(doc=73)
        0.32 = coord(8/25)
    
  4. Bateman, J.: Modelling the importance of end-user relevance criteria (1999) 0.26
    0.262886 = sum of:
      0.262886 = product of:
        0.7302388 = sum of:
          0.026810966 = weight(abstract_txt:selected in 1607) [ClassicSimilarity], result of:
            0.026810966 = score(doc=1607,freq=1.0), product of:
              0.093583934 = queryWeight, product of:
                1.0201422 = boost
                5.2386947 = idf(docFreq=623, maxDocs=43254)
                0.017511263 = queryNorm
              0.28649113 = fieldWeight in 1607, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.2386947 = idf(docFreq=623, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.02794727 = weight(abstract_txt:were in 1607) [ClassicSimilarity], result of:
            0.02794727 = score(doc=1607,freq=4.0), product of:
              0.06937926 = queryWeight, product of:
                1.0757728 = boost
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.017511263 = queryNorm
              0.40281877 = fieldWeight in 1607, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.034593996 = weight(abstract_txt:most in 1607) [ClassicSimilarity], result of:
            0.034593996 = score(doc=1607,freq=4.0), product of:
              0.07998414 = queryWeight, product of:
                1.1550683 = boost
                3.9543834 = idf(docFreq=2253, maxDocs=43254)
                0.017511263 = queryNorm
              0.43251067 = fieldWeight in 1607, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.9543834 = idf(docFreq=2253, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.02997882 = weight(abstract_txt:important in 1607) [ClassicSimilarity], result of:
            0.02997882 = score(doc=1607,freq=2.0), product of:
              0.0915988 = queryWeight, product of:
                1.2360915 = boost
                4.2317667 = idf(docFreq=1707, maxDocs=43254)
                0.017511263 = queryNorm
              0.32728398 = fieldWeight in 1607, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.2317667 = idf(docFreq=1707, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.024026863 = weight(abstract_txt:their in 1607) [ClassicSimilarity], result of:
            0.024026863 = score(doc=1607,freq=4.0), product of:
              0.06904216 = queryWeight, product of:
                1.2391739 = boost
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.017511263 = queryNorm
              0.34800276 = fieldWeight in 1607, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.01534601 = weight(abstract_txt:study in 1607) [ClassicSimilarity], result of:
            0.01534601 = score(doc=1607,freq=1.0), product of:
              0.08128305 = queryWeight, product of:
                1.3445442 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.017511263 = queryNorm
              0.18879718 = fieldWeight in 1607, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.106195435 = weight(abstract_txt:relevance in 1607) [ClassicSimilarity], result of:
            0.106195435 = score(doc=1607,freq=10.0), product of:
              0.12447839 = queryWeight, product of:
                1.4409626 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.017511263 = queryNorm
              0.8531234 = fieldWeight in 1607, product of:
                3.1622777 = tf(freq=10.0), with freq of:
                  10.0 = termFreq=10.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.10561709 = weight(abstract_txt:topicality in 1607) [ClassicSimilarity], result of:
            0.10561709 = score(doc=1607,freq=1.0), product of:
              0.23342595 = queryWeight, product of:
                1.611145 = boost
                8.273647 = idf(docFreq=29, maxDocs=43254)
                0.017511263 = queryNorm
              0.4524651 = fieldWeight in 1607, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.273647 = idf(docFreq=29, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
          0.35972235 = weight(abstract_txt:criteria in 1607) [ClassicSimilarity], result of:
            0.35972235 = score(doc=1607,freq=10.0), product of:
              0.37238652 = queryWeight, product of:
                3.807072 = boost
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.017511263 = queryNorm
              0.96599185 = fieldWeight in 1607, product of:
                3.1622777 = tf(freq=10.0), with freq of:
                  10.0 = termFreq=10.0
                5.5858 = idf(docFreq=440, maxDocs=43254)
                0.0546875 = fieldNorm(doc=1607)
        0.36 = coord(9/25)
    
  5. Ng, K.B.; Kantor, P.B.; Strzalkowski, T.; Wacholder, N.; Tang, R.; Bai, B.; Rittman,; Song, P.; Sun, Y.: Automated judgment of document qualities (2006) 0.23
    0.23467429 = sum of:
      0.23467429 = product of:
        0.7333572 = sum of:
          0.028231004 = weight(abstract_txt:were in 1308) [ClassicSimilarity], result of:
            0.028231004 = score(doc=1308,freq=2.0), product of:
              0.06937926 = queryWeight, product of:
                1.0757728 = boost
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.017511263 = queryNorm
              0.4069084 = fieldWeight in 1308, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6829145 = idf(docFreq=2956, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.030283183 = weight(abstract_txt:important in 1308) [ClassicSimilarity], result of:
            0.030283183 = score(doc=1308,freq=1.0), product of:
              0.0915988 = queryWeight, product of:
                1.2360915 = boost
                4.2317667 = idf(docFreq=1707, maxDocs=43254)
                0.017511263 = queryNorm
              0.33060676 = fieldWeight in 1308, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.2317667 = idf(docFreq=1707, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.017162045 = weight(abstract_txt:their in 1308) [ClassicSimilarity], result of:
            0.017162045 = score(doc=1308,freq=1.0), product of:
              0.06904216 = queryWeight, product of:
                1.2391739 = boost
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.017511263 = queryNorm
              0.24857341 = fieldWeight in 1308, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.1817396 = idf(docFreq=4880, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.062838964 = weight(abstract_txt:document in 1308) [ClassicSimilarity], result of:
            0.062838964 = score(doc=1308,freq=4.0), product of:
              0.09387605 = queryWeight, product of:
                1.2513624 = boost
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.017511263 = queryNorm
              0.6693823 = fieldWeight in 1308, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.2840466 = idf(docFreq=1620, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.021922871 = weight(abstract_txt:study in 1308) [ClassicSimilarity], result of:
            0.021922871 = score(doc=1308,freq=1.0), product of:
              0.08128305 = queryWeight, product of:
                1.3445442 = boost
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.017511263 = queryNorm
              0.26971024 = fieldWeight in 1308, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4522913 = idf(docFreq=3723, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.047974207 = weight(abstract_txt:relevance in 1308) [ClassicSimilarity], result of:
            0.047974207 = score(doc=1308,freq=1.0), product of:
              0.12447839 = queryWeight, product of:
                1.4409626 = boost
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.017511263 = queryNorm
              0.38540187 = fieldWeight in 1308, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.933144 = idf(docFreq=846, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.101476215 = weight(abstract_txt:stages in 1308) [ClassicSimilarity], result of:
            0.101476215 = score(doc=1308,freq=1.0), product of:
              0.2051154 = queryWeight, product of:
                1.849715 = boost
                6.332511 = idf(docFreq=208, maxDocs=43254)
                0.017511263 = queryNorm
              0.49472743 = fieldWeight in 1308, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.332511 = idf(docFreq=208, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
          0.4234687 = weight(abstract_txt:stage in 1308) [ClassicSimilarity], result of:
            0.4234687 = score(doc=1308,freq=3.0), product of:
              0.5111937 = queryWeight, product of:
                4.7685113 = boost
                6.1218853 = idf(docFreq=257, maxDocs=43254)
                0.017511263 = queryNorm
              0.8283919 = fieldWeight in 1308, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.1218853 = idf(docFreq=257, maxDocs=43254)
                0.078125 = fieldNorm(doc=1308)
        0.32 = coord(8/25)