Document (#30202)

Author

Maglaughlin, K.L.
Sonnenwald, D.H.

Title

User perspectives an relevance criteria : a comparison among relevant, partially relevant, and not-relevant judgements

Source

Journal of the American Society for Information Science and technology. 53(2002) no.5, S.327-342

Year

2002

Abstract

In this issue Maglaughin and Sonnenwald provided 12 graduate students with searches related to the student's work and asked them to judge the twenty most recent retrieved representations by highlighting passages thought to contribute to relevance, marking out passages detracting from relevance, and providing a relevant, partially relevant or relevant judgement on each. By recorded interview they were asked about how these decisions were made and to describe the three classes of judgement. The union of criteria identified in past studies did not seem to fully capture the information supplied so a new set was produced and coding agreement found to be adequate. Twenty-nine criteria were identified and grouped into six categories based upon the focus of the criterion. Multiple criteria are used for most judgements, and most criteria may have either a positive or negative effect. Content was the most frequently mentioned criterion.

Theme

Retrievalstudien
Benutzerstudien

Similar documents (author)

1. Sonnenwald, D.H.: Scientific collaboration (2007) 5.71

   5.7074614 = sum of:
     5.7074614 = weight(author_txt:sonnenwald in 3179) [ClassicSimilarity], result of:
       5.7074614 = fieldWeight in 3179, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.131938 = idf(docFreq=12, maxDocs=44218)
         0.625 = fieldNorm(doc=3179)
   

2. Sonnenwald, D.H.: Evolving perspectives of human information behaviour : contexts, situations, social networks and information horizons (1999) 5.71

   5.7074614 = sum of:
     5.7074614 = weight(author_txt:sonnenwald in 293) [ClassicSimilarity], result of:
       5.7074614 = fieldWeight in 293, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.131938 = idf(docFreq=12, maxDocs=44218)
         0.625 = fieldNorm(doc=293)
   

3. Sonnenwald, D.H.; Lievrouw, L.A.: Collaboration during the design process : a case study of communication, information behaviour, and project performance (1997) 4.57

   4.565969 = sum of:
     4.565969 = weight(author_txt:sonnenwald in 309) [ClassicSimilarity], result of:
       4.565969 = fieldWeight in 309, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.131938 = idf(docFreq=12, maxDocs=44218)
         0.5 = fieldNorm(doc=309)
   

4. Iivonen, M.; Sonnenwald, D.H.: Navigointi hekutermeja valittaessa (1997) 4.57

   4.565969 = sum of:
     4.565969 = weight(author_txt:sonnenwald in 1667) [ClassicSimilarity], result of:
       4.565969 = fieldWeight in 1667, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.131938 = idf(docFreq=12, maxDocs=44218)
         0.5 = fieldNorm(doc=1667)
   

5. Iivonen, M.; Sonnenwald, D.H.: From translation to navigation of different discourses : a model of search term selection during the pre-online stage of the search process (1998) 4.57

   4.565969 = sum of:
     4.565969 = weight(author_txt:sonnenwald in 2881) [ClassicSimilarity], result of:
       4.565969 = fieldWeight in 2881, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.131938 = idf(docFreq=12, maxDocs=44218)
         0.5 = fieldNorm(doc=2881)
   

Similar documents (content)

1. Tang, R.; Solomon, P.: Use of relevance criteria across stages of document evaluation : on the complementarity of experimental and naturalistic studies (2001) 0.27

   0.2657239 = sum of:
     0.2657239 = product of:
       0.83038723 = sum of:
         0.03271362 = weight(abstract_txt:were in 5213) [ClassicSimilarity], result of:
           0.03271362 = score(doc=5213,freq=3.0), product of:
             0.09410362 = queryWeight, product of:
               1.6417304 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.015618197 = queryNorm
             0.34763405 = fieldWeight in 5213, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.060458206 = weight(abstract_txt:asked in 5213) [ClassicSimilarity], result of:
           0.060458206 = score(doc=5213,freq=1.0), product of:
             0.1785531 = queryWeight, product of:
               1.8464466 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.015618197 = queryNorm
             0.3386007 = fieldWeight in 5213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.06481734 = weight(abstract_txt:relevance in 5213) [ClassicSimilarity], result of:
           0.06481734 = score(doc=5213,freq=2.0), product of:
             0.16993329 = queryWeight, product of:
               2.2061646 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015618197 = queryNorm
             0.38142815 = fieldWeight in 5213, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.10525519 = weight(abstract_txt:judgements in 5213) [ClassicSimilarity], result of:
           0.10525519 = score(doc=5213,freq=1.0), product of:
             0.25840023 = queryWeight, product of:
               2.2212605 = boost
               7.448392 = idf(docFreq=69, maxDocs=44218)
               0.015618197 = queryNorm
             0.40733394 = fieldWeight in 5213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.448392 = idf(docFreq=69, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.11664462 = weight(abstract_txt:criterion in 5213) [ClassicSimilarity], result of:
           0.11664462 = score(doc=5213,freq=1.0), product of:
             0.2767199 = queryWeight, product of:
               2.2986522 = boost
               7.7079034 = idf(docFreq=53, maxDocs=44218)
               0.015618197 = queryNorm
             0.42152596 = fieldWeight in 5213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7079034 = idf(docFreq=53, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.05411976 = weight(abstract_txt:most in 5213) [ClassicSimilarity], result of:
           0.05411976 = score(doc=5213,freq=3.0), product of:
             0.14487854 = queryWeight, product of:
               2.3521774 = boost
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.015618197 = queryNorm
             0.37355262 = fieldWeight in 5213, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.12619139 = weight(abstract_txt:judgement in 5213) [ClassicSimilarity], result of:
           0.12619139 = score(doc=5213,freq=1.0), product of:
             0.29161984 = queryWeight, product of:
               2.359726 = boost
               7.912698 = idf(docFreq=43, maxDocs=44218)
               0.015618197 = queryNorm
             0.43272567 = fieldWeight in 5213, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.912698 = idf(docFreq=43, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
         0.2701871 = weight(abstract_txt:criteria in 5213) [ClassicSimilarity], result of:
           0.2701871 = score(doc=5213,freq=6.0), product of:
             0.3618289 = queryWeight, product of:
               4.155992 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.015618197 = queryNorm
             0.7467262 = fieldWeight in 5213, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5213)
       0.32 = coord(8/25)
   

2. Taylor, A.; Zhang, X.; Amadio, W.J.: Examination of relevance criteria choices and the information search process (2009) 0.22

   0.21773607 = sum of:
     0.21773607 = product of:
       0.9072336 = sum of:
         0.034629047 = weight(abstract_txt:identified in 3608) [ClassicSimilarity], result of:
           0.034629047 = score(doc=3608,freq=1.0), product of:
             0.11265824 = queryWeight, product of:
               1.4666772 = boost
               4.9181023 = idf(docFreq=878, maxDocs=44218)
               0.015618197 = queryNorm
             0.3073814 = fieldWeight in 3608, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9181023 = idf(docFreq=878, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.043170784 = weight(abstract_txt:were in 3608) [ClassicSimilarity], result of:
           0.043170784 = score(doc=3608,freq=4.0), product of:
             0.09410362 = queryWeight, product of:
               1.6417304 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.015618197 = queryNorm
             0.45875797 = fieldWeight in 3608, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.16031219 = weight(abstract_txt:partially in 3608) [ClassicSimilarity], result of:
           0.16031219 = score(doc=3608,freq=2.0), product of:
             0.2483726 = queryWeight, product of:
               2.1777344 = boost
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.015618197 = queryNorm
             0.6454504 = fieldWeight in 3608, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.3024383 = idf(docFreq=80, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.1385853 = weight(abstract_txt:relevance in 3608) [ClassicSimilarity], result of:
           0.1385853 = score(doc=3608,freq=7.0), product of:
             0.16993329 = queryWeight, product of:
               2.2061646 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015618197 = queryNorm
             0.8155277 = fieldWeight in 3608, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.17398177 = weight(abstract_txt:relevant in 3608) [ClassicSimilarity], result of:
           0.17398177 = score(doc=3608,freq=4.0), product of:
             0.30025634 = queryWeight, product of:
               4.147246 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.015618197 = queryNorm
             0.5794441 = fieldWeight in 3608, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
         0.3565545 = weight(abstract_txt:criteria in 3608) [ClassicSimilarity], result of:
           0.3565545 = score(doc=3608,freq=8.0), product of:
             0.3618289 = queryWeight, product of:
               4.155992 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.015618197 = queryNorm
             0.98542297 = fieldWeight in 3608, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0625 = fieldNorm(doc=3608)
       0.24 = coord(6/25)
   

3. Crystal, A.; Greenberg, J.: Relevance criteria identified by health information users during Web searches (2006) 0.21

   0.20931673 = sum of:
     0.20931673 = product of:
       0.7475597 = sum of:
         0.054513946 = weight(abstract_txt:highlighting in 5909) [ClassicSimilarity], result of:
           0.054513946 = score(doc=5909,freq=1.0), product of:
             0.121003434 = queryWeight, product of:
               1.0748229 = boost
               7.208251 = idf(docFreq=88, maxDocs=44218)
               0.015618197 = queryNorm
             0.4505157 = fieldWeight in 5909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.208251 = idf(docFreq=88, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.059979267 = weight(abstract_txt:identified in 5909) [ClassicSimilarity], result of:
           0.059979267 = score(doc=5909,freq=3.0), product of:
             0.11265824 = queryWeight, product of:
               1.4666772 = boost
               4.9181023 = idf(docFreq=878, maxDocs=44218)
               0.015618197 = queryNorm
             0.5324002 = fieldWeight in 5909, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.9181023 = idf(docFreq=878, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.030526353 = weight(abstract_txt:were in 5909) [ClassicSimilarity], result of:
           0.030526353 = score(doc=5909,freq=2.0), product of:
             0.09410362 = queryWeight, product of:
               1.6417304 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.015618197 = queryNorm
             0.32439086 = fieldWeight in 5909, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.06909509 = weight(abstract_txt:asked in 5909) [ClassicSimilarity], result of:
           0.06909509 = score(doc=5909,freq=1.0), product of:
             0.1785531 = queryWeight, product of:
               1.8464466 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.015618197 = queryNorm
             0.38697222 = fieldWeight in 5909, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.10476065 = weight(abstract_txt:relevance in 5909) [ClassicSimilarity], result of:
           0.10476065 = score(doc=5909,freq=4.0), product of:
             0.16993329 = queryWeight, product of:
               2.2061646 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015618197 = queryNorm
             0.616481 = fieldWeight in 5909, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.05050125 = weight(abstract_txt:most in 5909) [ClassicSimilarity], result of:
           0.05050125 = score(doc=5909,freq=2.0), product of:
             0.14487854 = queryWeight, product of:
               2.3521774 = boost
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.015618197 = queryNorm
             0.3485765 = fieldWeight in 5909, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
         0.37818316 = weight(abstract_txt:criteria in 5909) [ClassicSimilarity], result of:
           0.37818316 = score(doc=5909,freq=9.0), product of:
             0.3618289 = queryWeight, product of:
               4.155992 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.015618197 = queryNorm
             1.0451989 = fieldWeight in 5909, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0625 = fieldNorm(doc=5909)
       0.28 = coord(7/25)
   

4. Janes, J.W.: ¬The binary nature of continous relevance judgements : a study of users' perceptions (1991) 0.17

   0.16588719 = sum of:
     0.16588719 = product of:
       0.6911966 = sum of:
         0.10816405 = weight(abstract_txt:judge in 4845) [ClassicSimilarity], result of:
           0.10816405 = score(doc=4845,freq=1.0), product of:
             0.14580992 = queryWeight, product of:
               1.179863 = boost
               7.912698 = idf(docFreq=43, maxDocs=44218)
               0.015618197 = queryNorm
             0.74181545 = fieldWeight in 4845, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.912698 = idf(docFreq=43, maxDocs=44218)
               0.09375 = fieldNorm(doc=4845)
         0.03237809 = weight(abstract_txt:were in 4845) [ClassicSimilarity], result of:
           0.03237809 = score(doc=4845,freq=1.0), product of:
             0.09410362 = queryWeight, product of:
               1.6417304 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.015618197 = queryNorm
             0.34406847 = fieldWeight in 4845, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.09375 = fieldNorm(doc=4845)
         0.10364264 = weight(abstract_txt:asked in 4845) [ClassicSimilarity], result of:
           0.10364264 = score(doc=4845,freq=1.0), product of:
             0.1785531 = queryWeight, product of:
               1.8464466 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.015618197 = queryNorm
             0.58045834 = fieldWeight in 4845, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.09375 = fieldNorm(doc=4845)
         0.13608807 = weight(abstract_txt:relevance in 4845) [ClassicSimilarity], result of:
           0.13608807 = score(doc=4845,freq=3.0), product of:
             0.16993329 = queryWeight, product of:
               2.2061646 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015618197 = queryNorm
             0.80083233 = fieldWeight in 4845, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.09375 = fieldNorm(doc=4845)
         0.18043746 = weight(abstract_txt:judgements in 4845) [ClassicSimilarity], result of:
           0.18043746 = score(doc=4845,freq=1.0), product of:
             0.25840023 = queryWeight, product of:
               2.2212605 = boost
               7.448392 = idf(docFreq=69, maxDocs=44218)
               0.015618197 = queryNorm
             0.6982868 = fieldWeight in 4845, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.448392 = idf(docFreq=69, maxDocs=44218)
               0.09375 = fieldNorm(doc=4845)
         0.13048632 = weight(abstract_txt:relevant in 4845) [ClassicSimilarity], result of:
           0.13048632 = score(doc=4845,freq=1.0), product of:
             0.30025634 = queryWeight, product of:
               4.147246 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.015618197 = queryNorm
             0.43458307 = fieldWeight in 4845, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.09375 = fieldNorm(doc=4845)
       0.24 = coord(6/25)
   

5. Choi, Y.; Rasmussen, E.M.: Users' relevance criteria in image retrieval in American history (2002) 0.16

   0.1593158 = sum of:
     0.1593158 = product of:
       0.66381586 = sum of:
         0.026710559 = weight(abstract_txt:were in 2592) [ClassicSimilarity], result of:
           0.026710559 = score(doc=2592,freq=2.0), product of:
             0.09410362 = queryWeight, product of:
               1.6417304 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.015618197 = queryNorm
             0.283842 = fieldWeight in 2592, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.12126213 = weight(abstract_txt:relevance in 2592) [ClassicSimilarity], result of:
           0.12126213 = score(doc=2592,freq=7.0), product of:
             0.16993329 = queryWeight, product of:
               2.2061646 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.015618197 = queryNorm
             0.7135867 = fieldWeight in 2592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.11664462 = weight(abstract_txt:criterion in 2592) [ClassicSimilarity], result of:
           0.11664462 = score(doc=2592,freq=1.0), product of:
             0.2767199 = queryWeight, product of:
               2.2986522 = boost
               7.7079034 = idf(docFreq=53, maxDocs=44218)
               0.015618197 = queryNorm
             0.42152596 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7079034 = idf(docFreq=53, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.031246057 = weight(abstract_txt:most in 2592) [ClassicSimilarity], result of:
           0.031246057 = score(doc=2592,freq=1.0), product of:
             0.14487854 = queryWeight, product of:
               2.3521774 = boost
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.015618197 = queryNorm
             0.2156707 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.943693 = idf(docFreq=2328, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.076117024 = weight(abstract_txt:relevant in 2592) [ClassicSimilarity], result of:
           0.076117024 = score(doc=2592,freq=1.0), product of:
             0.30025634 = queryWeight, product of:
               4.147246 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.015618197 = queryNorm
             0.2535068 = fieldWeight in 2592, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
         0.29183543 = weight(abstract_txt:criteria in 2592) [ClassicSimilarity], result of:
           0.29183543 = score(doc=2592,freq=7.0), product of:
             0.3618289 = queryWeight, product of:
               4.155992 = boost
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.015618197 = queryNorm
             0.80655646 = fieldWeight in 2592, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.574394 = idf(docFreq=455, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2592)
       0.24 = coord(6/25)