Document (#30214)

Author

Tang, R.
Solomon, P.

Title

Use of relevance criteria across stages of document evaluation : on the complementarity of experimental and naturalistic studies

Source

Journal of the American Society for Information Science and technology. 52(2001) no.8, S.676-687

Year

2001

Abstract

Tang and Solomon, based upon their review of the history of topical and other than topical criteria in relevance evaluation, decide to look at a two stage model where judgements are first made on surrogate records and then on full document text to determine if a criteria shift takes place and if so in what manner and to what degree. Both a controlled experiment and a naturalistic study were used to study the staging of relevance judgement criteria. In the controlled environment 90 undergraduate Psychology students were instructed to choose papers that would help them meet an assignment from 20 preselected papers on broader topic that included that assigned. They first selected on the basis of citation and abstract, then read the papers, and in each process filled out a questionnaire on the importance of each of 15 criteria at each stage of the two-stage process. In the naturalistic study 9 Ph.D. Psychology students conducted literature searches to support their own research and were asked to think aloud while making their decisions from retrieved surrogates, and later filled out a questionnaire while reading those materials that they selected and then interviewed at the end of the process. Apparently understandability is important at both stages. Importance increased at stage two. Cognitive criteria do not all follow the same pattern across stages. The controlled group thought quality of information was most important in stage one and topicality most important in stage 2. In the naturalistic study topicality was most frequent for stage one and research structure for stage two. A classification of criteria by their functionality is suggested as a better approach. First a division as to whether a criterion is objectively associated with the document as opposed to being subjectively associated with a person's expectations; then a division based on primary (essential) or secondary (for assistance) status.

Theme

Benutzerstudien

Similar documents (author)

1. Tang, R.; Solomon, P.: Toward an understanding of the dynamics of relevance judgement : an analysis of one person's search behavior (1998) 5.90

   5.8989305 = sum of:
     5.8989305 = sum of:
       2.570637 = weight(author_txt:tang in 3268) [ClassicSimilarity], result of:
         2.570637 = score(doc=3268,freq=1.0), product of:
           0.6440024 = queryWeight, product of:
             7.983315 = idf(docFreq=40, maxDocs=44218)
             0.08066854 = queryNorm
           3.9916575 = fieldWeight in 3268, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.983315 = idf(docFreq=40, maxDocs=44218)
             0.5 = fieldNorm(doc=3268)
       3.3282936 = weight(author_txt:solomon in 3268) [ClassicSimilarity], result of:
         3.3282936 = score(doc=3268,freq=1.0), product of:
           0.7650234 = queryWeight, product of:
             1.0899175 = boost
             8.701155 = idf(docFreq=19, maxDocs=44218)
             0.08066854 = queryNorm
           4.3505774 = fieldWeight in 3268, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.701155 = idf(docFreq=19, maxDocs=44218)
             0.5 = fieldNorm(doc=3268)
   

2. Solomon, P.: Children's information retrieval behavior : a case analysis of an OPAC (1993) 2.08

   2.0801835 = sum of:
     2.0801835 = product of:
       4.160367 = sum of:
         4.160367 = weight(author_txt:solomon in 4349) [ClassicSimilarity], result of:
           4.160367 = score(doc=4349,freq=1.0), product of:
             0.7650234 = queryWeight, product of:
               1.0899175 = boost
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.08066854 = queryNorm
             5.438222 = fieldWeight in 4349, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.625 = fieldNorm(doc=4349)
       0.5 = coord(1/2)
   

3. Solomon, P.: On the dynamics of information system use : from novice to? (1992) 2.08

   2.0801835 = sum of:
     2.0801835 = product of:
       4.160367 = sum of:
         4.160367 = weight(author_txt:solomon in 4519) [ClassicSimilarity], result of:
           4.160367 = score(doc=4519,freq=1.0), product of:
             0.7650234 = queryWeight, product of:
               1.0899175 = boost
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.08066854 = queryNorm
             5.438222 = fieldWeight in 4519, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.625 = fieldNorm(doc=4519)
       0.5 = coord(1/2)
   

4. Solomon, P.: Children, technology, and instruction : a case study of elementary school children using an online public access catalog (OPAC) (1994) 2.08

   2.0801835 = sum of:
     2.0801835 = product of:
       4.160367 = sum of:
         4.160367 = weight(author_txt:solomon in 1423) [ClassicSimilarity], result of:
           4.160367 = score(doc=1423,freq=1.0), product of:
             0.7650234 = queryWeight, product of:
               1.0899175 = boost
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.08066854 = queryNorm
             5.438222 = fieldWeight in 1423, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.625 = fieldNorm(doc=1423)
       0.5 = coord(1/2)
   

5. Solomon, P.: Dicovering information bahavior in sense making : I. time and timing (1997) 2.08

   2.0801835 = sum of:
     2.0801835 = product of:
       4.160367 = sum of:
         4.160367 = weight(author_txt:solomon in 957) [ClassicSimilarity], result of:
           4.160367 = score(doc=957,freq=1.0), product of:
             0.7650234 = queryWeight, product of:
               1.0899175 = boost
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.08066854 = queryNorm
             5.438222 = fieldWeight in 957, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.701155 = idf(docFreq=19, maxDocs=44218)
               0.625 = fieldNorm(doc=957)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Wang, P.; White, M.D.: ¬A cognitive model of document use during a research project : Study II: Decisions at the reading and citing stages (1999) 0.35

   0.35377163 = sum of:
     0.35377163 = product of:
       0.8040264 = sum of:
         0.030669114 = weight(abstract_txt:selected in 2940) [ClassicSimilarity], result of:
           0.030669114 = score(doc=2940,freq=1.0), product of:
             0.09378398 = queryWeight, product of:
               1.0262673 = boost
               5.232299 = idf(docFreq=641, maxDocs=44218)
               0.017465284 = queryNorm
             0.32701868 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.232299 = idf(docFreq=641, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.015875803 = weight(abstract_txt:were in 2940) [ClassicSimilarity], result of:
           0.015875803 = score(doc=2940,freq=1.0), product of:
             0.06921211 = queryWeight, product of:
               1.0797739 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017465284 = queryNorm
             0.22937898 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.030193975 = weight(abstract_txt:process in 2940) [ClassicSimilarity], result of:
           0.030193975 = score(doc=2940,freq=2.0), product of:
             0.08432606 = queryWeight, product of:
               1.1918529 = boost
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.017465284 = queryNorm
             0.3580622 = fieldWeight in 2940, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.022439266 = weight(abstract_txt:each in 2940) [ClassicSimilarity], result of:
           0.022439266 = score(doc=2940,freq=1.0), product of:
             0.087169394 = queryWeight, product of:
               1.2117801 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.017465284 = queryNorm
             0.25742137 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.023254316 = weight(abstract_txt:first in 2940) [ClassicSimilarity], result of:
           0.023254316 = score(doc=2940,freq=1.0), product of:
             0.08926763 = queryWeight, product of:
               1.2262775 = boost
               4.168018 = idf(docFreq=1860, maxDocs=44218)
               0.017465284 = queryNorm
             0.26050112 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.168018 = idf(docFreq=1860, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.013505431 = weight(abstract_txt:their in 2940) [ClassicSimilarity], result of:
           0.013505431 = score(doc=2940,freq=1.0), product of:
             0.06839276 = queryWeight, product of:
               1.2394135 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.017465284 = queryNorm
             0.19746871 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.05680201 = weight(abstract_txt:document in 2940) [ClassicSimilarity], result of:
           0.05680201 = score(doc=2940,freq=5.0), product of:
             0.094684176 = queryWeight, product of:
               1.2629335 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017465284 = queryNorm
             0.59991026 = fieldWeight in 2940, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.045470536 = weight(abstract_txt:study in 2940) [ClassicSimilarity], result of:
           0.045470536 = score(doc=2940,freq=7.0), product of:
             0.08031412 = queryWeight, product of:
               1.3430961 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.017465284 = queryNorm
             0.56615865 = fieldWeight in 2940, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.08025512 = weight(abstract_txt:stages in 2940) [ClassicSimilarity], result of:
           0.08025512 = score(doc=2940,freq=1.0), product of:
             0.2038638 = queryWeight, product of:
               1.8531554 = boost
               6.2987247 = idf(docFreq=220, maxDocs=44218)
               0.017465284 = queryNorm
             0.3936703 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987247 = idf(docFreq=220, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.29024833 = weight(abstract_txt:criteria in 2940) [ClassicSimilarity], result of:
           0.29024833 = score(doc=2940,freq=5.0), product of:
             0.37256926 = queryWeight, product of:
               3.8267815 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.017465284 = queryNorm
             0.7790453 = fieldWeight in 2940, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
         0.19531257 = weight(abstract_txt:stage in 2940) [ClassicSimilarity], result of:
           0.19531257 = score(doc=2940,freq=1.0), product of:
             0.51148707 = queryWeight, product of:
               4.793398 = boost
               6.1096387 = idf(docFreq=266, maxDocs=44218)
               0.017465284 = queryNorm
             0.38185242 = fieldWeight in 2940, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1096387 = idf(docFreq=266, maxDocs=44218)
               0.0625 = fieldNorm(doc=2940)
       0.44 = coord(11/25)
   

2. Taylor, A.; Zhang, X.; Amadio, W.J.: Examination of relevance criteria choices and the information search process (2009) 0.35

   0.3509118 = sum of:
     0.3509118 = product of:
       0.974755 = sum of:
         0.028373947 = weight(abstract_txt:importance in 3608) [ClassicSimilarity], result of:
           0.028373947 = score(doc=3608,freq=1.0), product of:
             0.08904462 = queryWeight, product of:
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.017465284 = queryNorm
             0.31864864 = fieldWeight in 3608, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.031751607 = weight(abstract_txt:were in 3608) [ClassicSimilarity], result of:
           0.031751607 = score(doc=3608,freq=4.0), product of:
             0.06921211 = queryWeight, product of:
               1.0797739 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017465284 = queryNorm
             0.45875797 = fieldWeight in 3608, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.047740865 = weight(abstract_txt:process in 3608) [ClassicSimilarity], result of:
           0.047740865 = score(doc=3608,freq=5.0), product of:
             0.08432606 = queryWeight, product of:
               1.1918529 = boost
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.017465284 = queryNorm
             0.5661461 = fieldWeight in 3608, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.029767456 = weight(abstract_txt:study in 3608) [ClassicSimilarity], result of:
           0.029767456 = score(doc=3608,freq=3.0), product of:
             0.08031412 = queryWeight, product of:
               1.3430961 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.017465284 = queryNorm
             0.37063786 = fieldWeight in 3608, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.10192787 = weight(abstract_txt:relevance in 3608) [ClassicSimilarity], result of:
           0.10192787 = score(doc=3608,freq=7.0), product of:
             0.124983944 = queryWeight, product of:
               1.4510049 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017465284 = queryNorm
             0.8155277 = fieldWeight in 3608, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.05195509 = weight(abstract_txt:controlled in 3608) [ClassicSimilarity], result of:
           0.05195509 = score(doc=3608,freq=1.0), product of:
             0.15256105 = queryWeight, product of:
               1.603111 = boost
               5.4488444 = idf(docFreq=516, maxDocs=44218)
               0.017465284 = queryNorm
             0.34055278 = fieldWeight in 3608, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4488444 = idf(docFreq=516, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.12078727 = weight(abstract_txt:topicality in 3608) [ClassicSimilarity], result of:
           0.12078727 = score(doc=3608,freq=1.0), product of:
             0.23388837 = queryWeight, product of:
               1.6206919 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.017465284 = queryNorm
             0.5164313 = fieldWeight in 3608, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.3671383 = weight(abstract_txt:criteria in 3608) [ClassicSimilarity], result of:
           0.3671383 = score(doc=3608,freq=8.0), product of:
             0.37256926 = queryWeight, product of:
               3.8267815 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.017465284 = queryNorm
             0.98542297 = fieldWeight in 3608, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.19531257 = weight(abstract_txt:stage in 3608) [ClassicSimilarity], result of:
           0.19531257 = score(doc=3608,freq=1.0), product of:
             0.51148707 = queryWeight, product of:
               4.793398 = boost
               6.1096387 = idf(docFreq=266, maxDocs=44218)
               0.017465284 = queryNorm
             0.38185242 = fieldWeight in 3608, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1096387 = idf(docFreq=266, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
       0.36 = coord(9/25)
   

3. Choi, Y.; Rasmussen, E.M.: Users' relevance criteria in image retrieval in American history (2002) 0.34

   0.33700287 = sum of:
     0.33700287 = product of:
       0.76591563 = sum of:
         0.024827205 = weight(abstract_txt:importance in 2592) [ClassicSimilarity], result of:
           0.024827205 = score(doc=2592,freq=1.0), product of:
             0.08904462 = queryWeight, product of:
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.017465284 = queryNorm
             0.27881756 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.019645303 = weight(abstract_txt:were in 2592) [ClassicSimilarity], result of:
           0.019645303 = score(doc=2592,freq=2.0), product of:
             0.06921211 = queryWeight, product of:
               1.0797739 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017465284 = queryNorm
             0.283842 = fieldWeight in 2592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.01723583 = weight(abstract_txt:most in 2592) [ClassicSimilarity], result of:
           0.01723583 = score(doc=2592,freq=1.0), product of:
             0.07991735 = queryWeight, product of:
               1.1602787 = boost
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.017465284 = queryNorm
             0.2156707 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.019634357 = weight(abstract_txt:each in 2592) [ClassicSimilarity], result of:
           0.019634357 = score(doc=2592,freq=1.0), product of:
             0.087169394 = queryWeight, product of:
               1.2117801 = boost
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.017465284 = queryNorm
             0.2252437 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.118742 = idf(docFreq=1954, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.011817251 = weight(abstract_txt:their in 2592) [ClassicSimilarity], result of:
           0.011817251 = score(doc=2592,freq=1.0), product of:
             0.06839276 = queryWeight, product of:
               1.2394135 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.017465284 = queryNorm
             0.17278512 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.02975494 = weight(abstract_txt:important in 2592) [ClassicSimilarity], result of:
           0.02975494 = score(doc=2592,freq=2.0), product of:
             0.0912814 = queryWeight, product of:
               1.2400321 = boost
               4.2147684 = idf(docFreq=1775, maxDocs=44218)
               0.017465284 = queryNorm
             0.32596937 = fieldWeight in 2592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2147684 = idf(docFreq=1775, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.03362592 = weight(abstract_txt:study in 2592) [ClassicSimilarity], result of:
           0.03362592 = score(doc=2592,freq=5.0), product of:
             0.08031412 = queryWeight, product of:
               1.3430961 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.017465284 = queryNorm
             0.41868 = fieldWeight in 2592, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.08918688 = weight(abstract_txt:relevance in 2592) [ClassicSimilarity], result of:
           0.08918688 = score(doc=2592,freq=7.0), product of:
             0.124983944 = queryWeight, product of:
               1.4510049 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017465284 = queryNorm
             0.7135867 = fieldWeight in 2592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.14946662 = weight(abstract_txt:topicality in 2592) [ClassicSimilarity], result of:
           0.14946662 = score(doc=2592,freq=2.0), product of:
             0.23388837 = queryWeight, product of:
               1.6206919 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.017465284 = queryNorm
             0.6390511 = fieldWeight in 2592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.07022323 = weight(abstract_txt:stages in 2592) [ClassicSimilarity], result of:
           0.07022323 = score(doc=2592,freq=1.0), product of:
             0.2038638 = queryWeight, product of:
               1.8531554 = boost
               6.2987247 = idf(docFreq=220, maxDocs=44218)
               0.017465284 = queryNorm
             0.3444615 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987247 = idf(docFreq=220, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.30049816 = weight(abstract_txt:criteria in 2592) [ClassicSimilarity], result of:
           0.30049816 = score(doc=2592,freq=7.0), product of:
             0.37256926 = queryWeight, product of:
               3.8267815 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.017465284 = queryNorm
             0.80655646 = fieldWeight in 2592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
       0.44 = coord(11/25)
   

4. Bateman, J.: Modelling the importance of end-user relevance criteria (1999) 0.31

   0.3087883 = sum of:
     0.3087883 = product of:
       0.77197075 = sum of:
         0.04300198 = weight(abstract_txt:importance in 6606) [ClassicSimilarity], result of:
           0.04300198 = score(doc=6606,freq=3.0), product of:
             0.08904462 = queryWeight, product of:
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.017465284 = queryNorm
             0.4829262 = fieldWeight in 6606, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.026835473 = weight(abstract_txt:selected in 6606) [ClassicSimilarity], result of:
           0.026835473 = score(doc=6606,freq=1.0), product of:
             0.09378398 = queryWeight, product of:
               1.0262673 = boost
               5.232299 = idf(docFreq=641, maxDocs=44218)
               0.017465284 = queryNorm
             0.28614134 = fieldWeight in 6606, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.232299 = idf(docFreq=641, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.027782656 = weight(abstract_txt:were in 6606) [ClassicSimilarity], result of:
           0.027782656 = score(doc=6606,freq=4.0), product of:
             0.06921211 = queryWeight, product of:
               1.0797739 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017465284 = queryNorm
             0.40141323 = fieldWeight in 6606, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.03447166 = weight(abstract_txt:most in 6606) [ClassicSimilarity], result of:
           0.03447166 = score(doc=6606,freq=4.0), product of:
             0.07991735 = queryWeight, product of:
               1.1602787 = boost
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.017465284 = queryNorm
             0.4313414 = fieldWeight in 6606, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.023634503 = weight(abstract_txt:their in 6606) [ClassicSimilarity], result of:
           0.023634503 = score(doc=6606,freq=4.0), product of:
             0.06839276 = queryWeight, product of:
               1.2394135 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.017465284 = queryNorm
             0.34557024 = fieldWeight in 6606, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.02975494 = weight(abstract_txt:important in 6606) [ClassicSimilarity], result of:
           0.02975494 = score(doc=6606,freq=2.0), product of:
             0.0912814 = queryWeight, product of:
               1.2400321 = boost
               4.2147684 = idf(docFreq=1775, maxDocs=44218)
               0.017465284 = queryNorm
             0.32596937 = fieldWeight in 6606, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2147684 = idf(docFreq=1775, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.015037968 = weight(abstract_txt:study in 6606) [ClassicSimilarity], result of:
           0.015037968 = score(doc=6606,freq=1.0), product of:
             0.08031412 = queryWeight, product of:
               1.3430961 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.017465284 = queryNorm
             0.1872394 = fieldWeight in 6606, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.10659872 = weight(abstract_txt:relevance in 6606) [ClassicSimilarity], result of:
           0.10659872 = score(doc=6606,freq=10.0), product of:
             0.124983944 = queryWeight, product of:
               1.4510049 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017465284 = queryNorm
             0.8528993 = fieldWeight in 6606, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.10568886 = weight(abstract_txt:topicality in 6606) [ClassicSimilarity], result of:
           0.10568886 = score(doc=6606,freq=1.0), product of:
             0.23388837 = queryWeight, product of:
               1.6206919 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.017465284 = queryNorm
             0.45187736 = fieldWeight in 6606, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
         0.35916397 = weight(abstract_txt:criteria in 6606) [ClassicSimilarity], result of:
           0.35916397 = score(doc=6606,freq=10.0), product of:
             0.37256926 = queryWeight, product of:
               3.8267815 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.017465284 = queryNorm
             0.96401936 = fieldWeight in 6606, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0546875 = fieldNorm(doc=6606)
       0.4 = coord(10/25)
   

5. Ng, K.B.; Kantor, P.B.; Strzalkowski, T.; Wacholder, N.; Tang, R.; Bai, B.; Rittman,; Song, P.; Sun, Y.: Automated judgment of document qualities (2006) 0.23

   0.23402636 = sum of:
     0.23402636 = product of:
       0.73133236 = sum of:
         0.02806472 = weight(abstract_txt:were in 182) [ClassicSimilarity], result of:
           0.02806472 = score(doc=182,freq=2.0), product of:
             0.06921211 = queryWeight, product of:
               1.0797739 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017465284 = queryNorm
             0.40548858 = fieldWeight in 182, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.016881788 = weight(abstract_txt:their in 182) [ClassicSimilarity], result of:
           0.016881788 = score(doc=182,freq=1.0), product of:
             0.06839276 = queryWeight, product of:
               1.2394135 = boost
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.017465284 = queryNorm
             0.24683589 = fieldWeight in 182, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.1594994 = idf(docFreq=5101, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.030057026 = weight(abstract_txt:important in 182) [ClassicSimilarity], result of:
           0.030057026 = score(doc=182,freq=1.0), product of:
             0.0912814 = queryWeight, product of:
               1.2400321 = boost
               4.2147684 = idf(docFreq=1775, maxDocs=44218)
               0.017465284 = queryNorm
             0.32927877 = fieldWeight in 182, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.2147684 = idf(docFreq=1775, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.06350657 = weight(abstract_txt:document in 182) [ClassicSimilarity], result of:
           0.06350657 = score(doc=182,freq=4.0), product of:
             0.094684176 = queryWeight, product of:
               1.2629335 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.017465284 = queryNorm
             0.67072004 = fieldWeight in 182, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.02148281 = weight(abstract_txt:study in 182) [ClassicSimilarity], result of:
           0.02148281 = score(doc=182,freq=1.0), product of:
             0.08031412 = queryWeight, product of:
               1.3430961 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.017465284 = queryNorm
             0.26748484 = fieldWeight in 182, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.04815639 = weight(abstract_txt:relevance in 182) [ClassicSimilarity], result of:
           0.04815639 = score(doc=182,freq=1.0), product of:
             0.124983944 = queryWeight, product of:
               1.4510049 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.017465284 = queryNorm
             0.38530064 = fieldWeight in 182, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.1003189 = weight(abstract_txt:stages in 182) [ClassicSimilarity], result of:
           0.1003189 = score(doc=182,freq=1.0), product of:
             0.2038638 = queryWeight, product of:
               1.8531554 = boost
               6.2987247 = idf(docFreq=220, maxDocs=44218)
               0.017465284 = queryNorm
             0.49208787 = fieldWeight in 182, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2987247 = idf(docFreq=220, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
         0.42286414 = weight(abstract_txt:stage in 182) [ClassicSimilarity], result of:
           0.42286414 = score(doc=182,freq=3.0), product of:
             0.51148707 = queryWeight, product of:
               4.793398 = boost
               6.1096387 = idf(docFreq=266, maxDocs=44218)
               0.017465284 = queryNorm
             0.8267348 = fieldWeight in 182, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1096387 = idf(docFreq=266, maxDocs=44218)
               0.078125 = fieldNorm(doc=182)
       0.32 = coord(8/25)