Document (#33879)

Author
Xu, Y.
Wang, D.
Title
Order effect in relevance judgment : mediation and causality
Source
Journal of the American Society for Information Science and Technology. 59(2008) no.8, S.1264-1275
Year
2008
Abstract
The order effect of relevance judgment refers to the different relevance perceptions of a document when it appears in different positions in a list. Although the order effect of relevance judgment has significant theoretical and practical implications, the extant literature is inconclusive regarding its existence and forming mechanisms. This study proposes a set of order effect forming mechanisms, including the learning effect, the subneed scheduling effect, and the cursoriness effect based on the conceptualization of dynamic relevance and the psychology of cognitive elaboration. Our empirical study indicates that in an interactive information retrieval setting, when a document list is reasonably long, order effects demonstrate a curvilinear pattern that conforms to the combined effect of the three mechanisms. Moreover, the curvilinear pattern of order effect could differ for documents of different relevance levels.
Theme
Retrievalalgorithmen

Similar documents (author)

  1. Wang, H.; Wang, C.: Ontologies for universal information systems (1995) 4.77
    4.7679567 = sum of:
      4.7679567 = weight(author_txt:wang in 3263) [ClassicSimilarity], result of:
        4.7679567 = fieldWeight in 3263, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          6.7429094 = idf(docFreq=136, maxDocs=42740)
          0.5 = fieldNorm(doc=3263)
    
  2. Wang, C.: ¬The online catalogue, subject access and user reactions : a review (1985) 4.21
    4.2143183 = sum of:
      4.2143183 = weight(author_txt:wang in 986) [ClassicSimilarity], result of:
        4.2143183 = fieldWeight in 986, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          6.7429094 = idf(docFreq=136, maxDocs=42740)
          0.625 = fieldNorm(doc=986)
    
  3. Wang, C.: Bibliometrics : a textbook (1990) 4.21
    4.2143183 = sum of:
      4.2143183 = weight(author_txt:wang in 5109) [ClassicSimilarity], result of:
        4.2143183 = fieldWeight in 5109, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          6.7429094 = idf(docFreq=136, maxDocs=42740)
          0.625 = fieldNorm(doc=5109)
    
  4. Wang, P.: Users' information needs at different stages of a research project : a cognitive view (1997) 4.21
    4.2143183 = sum of:
      4.2143183 = weight(author_txt:wang in 1321) [ClassicSimilarity], result of:
        4.2143183 = fieldWeight in 1321, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          6.7429094 = idf(docFreq=136, maxDocs=42740)
          0.625 = fieldNorm(doc=1321)
    
  5. Wang, D.: Cataloger appraises keyword searching in WorldCat (1997) 4.21
    4.2143183 = sum of:
      4.2143183 = weight(author_txt:wang in 2443) [ClassicSimilarity], result of:
        4.2143183 = fieldWeight in 2443, product of:
          1.0 = tf(freq=1.0), with freq of:
            1.0 = termFreq=1.0
          6.7429094 = idf(docFreq=136, maxDocs=42740)
          0.625 = fieldNorm(doc=2443)
    

Similar documents (content)

  1. Xu, Y.: Relevance judgment in epistemic and hedonic information searches (2007) 0.20
    0.20159595 = sum of:
      0.20159595 = product of:
        0.8399831 = sum of:
          0.023558877 = weight(abstract_txt:study in 2065) [ClassicSimilarity], result of:
            0.023558877 = score(doc=2065,freq=3.0), product of:
              0.050160877 = queryWeight, product of:
                1.0663863 = boost
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.013552261 = queryNorm
              0.46966636 = fieldWeight in 2065, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.0824666 = weight(abstract_txt:extant in 2065) [ClassicSimilarity], result of:
            0.0824666 = score(doc=2065,freq=1.0), product of:
              0.1323766 = queryWeight, product of:
                1.224962 = boost
                7.974011 = idf(docFreq=39, maxDocs=42740)
                0.013552261 = queryNorm
              0.6229696 = fieldWeight in 2065, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.974011 = idf(docFreq=39, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.023589917 = weight(abstract_txt:when in 2065) [ClassicSimilarity], result of:
            0.023589917 = score(doc=2065,freq=1.0), product of:
              0.072408035 = queryWeight, product of:
                1.281225 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.013552261 = queryNorm
              0.32579142 = fieldWeight in 2065, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.025517728 = weight(abstract_txt:document in 2065) [ClassicSimilarity], result of:
            0.025517728 = score(doc=2065,freq=1.0), product of:
              0.076301046 = queryWeight, product of:
                1.3152165 = boost
                4.280766 = idf(docFreq=1606, maxDocs=42740)
                0.013552261 = queryNorm
              0.33443484 = fieldWeight in 2065, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.280766 = idf(docFreq=1606, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.3331384 = weight(abstract_txt:judgment in 2065) [ClassicSimilarity], result of:
            0.3331384 = score(doc=2065,freq=3.0), product of:
              0.3357703 = queryWeight, product of:
                3.3790824 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.013552261 = queryNorm
              0.99216163 = fieldWeight in 2065, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
          0.35171154 = weight(abstract_txt:relevance in 2065) [ClassicSimilarity], result of:
            0.35171154 = score(doc=2065,freq=9.0), product of:
              0.3041257 = queryWeight, product of:
                4.547987 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.013552261 = queryNorm
              1.1564677 = fieldWeight in 2065, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.078125 = fieldNorm(doc=2065)
        0.24 = coord(6/25)
    
  2. Chu, H.: Factors affecting relevance judgment : a report from TREC Legal track (2011) 0.15
    0.15066679 = sum of:
      0.15066679 = product of:
        0.7533339 = sum of:
          0.021762758 = weight(abstract_txt:study in 1541) [ClassicSimilarity], result of:
            0.021762758 = score(doc=1541,freq=4.0), product of:
              0.050160877 = queryWeight, product of:
                1.0663863 = boost
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.013552261 = queryNorm
              0.4338592 = fieldWeight in 1541, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.0625 = fieldNorm(doc=1541)
          0.018871933 = weight(abstract_txt:when in 1541) [ClassicSimilarity], result of:
            0.018871933 = score(doc=1541,freq=1.0), product of:
              0.072408035 = queryWeight, product of:
                1.281225 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.013552261 = queryNorm
              0.26063314 = fieldWeight in 1541, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.0625 = fieldNorm(doc=1541)
          0.05757001 = weight(abstract_txt:list in 1541) [ClassicSimilarity], result of:
            0.05757001 = score(doc=1541,freq=2.0), product of:
              0.12088248 = queryWeight, product of:
                1.655441 = boost
                5.3881283 = idf(docFreq=530, maxDocs=42740)
                0.013552261 = queryNorm
              0.47624776 = fieldWeight in 1541, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.3881283 = idf(docFreq=530, maxDocs=42740)
                0.0625 = fieldNorm(doc=1541)
          0.34406388 = weight(abstract_txt:judgment in 1541) [ClassicSimilarity], result of:
            0.34406388 = score(doc=1541,freq=5.0), product of:
              0.3357703 = queryWeight, product of:
                3.3790824 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.013552261 = queryNorm
              1.0247002 = fieldWeight in 1541, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.0625 = fieldNorm(doc=1541)
          0.31106538 = weight(abstract_txt:relevance in 1541) [ClassicSimilarity], result of:
            0.31106538 = score(doc=1541,freq=11.0), product of:
              0.3041257 = queryWeight, product of:
                4.547987 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.013552261 = queryNorm
              1.0228184 = fieldWeight in 1541, product of:
                3.3166249 = tf(freq=11.0), with freq of:
                  11.0 = termFreq=11.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.0625 = fieldNorm(doc=1541)
        0.2 = coord(5/25)
    
  3. Huang, M.-h.; Wang, H.-y.: ¬The influence of document presentation order and number of documents judged an users' judgments of relevance (2004) 0.15
    0.15034506 = sum of:
      0.15034506 = product of:
        0.9396567 = sum of:
          0.046705656 = weight(abstract_txt:when in 3886) [ClassicSimilarity], result of:
            0.046705656 = score(doc=3886,freq=2.0), product of:
              0.072408035 = queryWeight, product of:
                1.281225 = boost
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.013552261 = queryNorm
              0.64503413 = fieldWeight in 3886, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.1701303 = idf(docFreq=1794, maxDocs=42740)
                0.109375 = fieldNorm(doc=3886)
          0.03572482 = weight(abstract_txt:document in 3886) [ClassicSimilarity], result of:
            0.03572482 = score(doc=3886,freq=1.0), product of:
              0.076301046 = queryWeight, product of:
                1.3152165 = boost
                4.280766 = idf(docFreq=1606, maxDocs=42740)
                0.013552261 = queryNorm
              0.4682088 = fieldWeight in 3886, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.280766 = idf(docFreq=1606, maxDocs=42740)
                0.109375 = fieldNorm(doc=3886)
          0.27216202 = weight(abstract_txt:order in 3886) [ClassicSimilarity], result of:
            0.27216202 = score(doc=3886,freq=5.0), product of:
              0.24916436 = queryWeight, product of:
                4.116571 = boost
                4.466204 = idf(docFreq=1334, maxDocs=42740)
                0.013552261 = queryNorm
              1.0922992 = fieldWeight in 3886, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                4.466204 = idf(docFreq=1334, maxDocs=42740)
                0.109375 = fieldNorm(doc=3886)
          0.5850642 = weight(abstract_txt:effect in 3886) [ClassicSimilarity], result of:
            0.5850642 = score(doc=3886,freq=3.0), product of:
              0.56326985 = queryWeight, product of:
                7.5804768 = boost
                5.482874 = idf(docFreq=482, maxDocs=42740)
                0.013552261 = queryNorm
              1.0386925 = fieldWeight in 3886, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.482874 = idf(docFreq=482, maxDocs=42740)
                0.109375 = fieldNorm(doc=3886)
        0.16 = coord(4/25)
    
  4. Balatsoukas, P.; Ruthven, I.: ¬An eye-tracking approach to the analysis of relevance judgments on the Web : the case of Google search engine (2012) 0.14
    0.13509361 = sum of:
      0.13509361 = product of:
        0.675468 = sum of:
          0.010881379 = weight(abstract_txt:study in 2380) [ClassicSimilarity], result of:
            0.010881379 = score(doc=2380,freq=1.0), product of:
              0.050160877 = queryWeight, product of:
                1.0663863 = boost
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.013552261 = queryNorm
              0.2169296 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.0625 = fieldNorm(doc=2380)
          0.15387005 = weight(abstract_txt:judgment in 2380) [ClassicSimilarity], result of:
            0.15387005 = score(doc=2380,freq=1.0), product of:
              0.3357703 = queryWeight, product of:
                3.3790824 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.013552261 = queryNorm
              0.45825982 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.0625 = fieldNorm(doc=2380)
          0.06955118 = weight(abstract_txt:order in 2380) [ClassicSimilarity], result of:
            0.06955118 = score(doc=2380,freq=1.0), product of:
              0.24916436 = queryWeight, product of:
                4.116571 = boost
                4.466204 = idf(docFreq=1334, maxDocs=42740)
                0.013552261 = queryNorm
              0.27913776 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.466204 = idf(docFreq=1334, maxDocs=42740)
                0.0625 = fieldNorm(doc=2380)
          0.24814433 = weight(abstract_txt:relevance in 2380) [ClassicSimilarity], result of:
            0.24814433 = score(doc=2380,freq=7.0), product of:
              0.3041257 = queryWeight, product of:
                4.547987 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.013552261 = queryNorm
              0.81592685 = fieldWeight in 2380, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.0625 = fieldNorm(doc=2380)
          0.1930211 = weight(abstract_txt:effect in 2380) [ClassicSimilarity], result of:
            0.1930211 = score(doc=2380,freq=1.0), product of:
              0.56326985 = queryWeight, product of:
                7.5804768 = boost
                5.482874 = idf(docFreq=482, maxDocs=42740)
                0.013552261 = queryNorm
              0.34267962 = fieldWeight in 2380, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.482874 = idf(docFreq=482, maxDocs=42740)
                0.0625 = fieldNorm(doc=2380)
        0.2 = coord(5/25)
    
  5. Xu, Y.; Yin, H.: Novelty and topicality in interactive information retrieval (2008) 0.13
    0.13190429 = sum of:
      0.13190429 = product of:
        0.65952146 = sum of:
          0.021762758 = weight(abstract_txt:study in 3356) [ClassicSimilarity], result of:
            0.021762758 = score(doc=3356,freq=4.0), product of:
              0.050160877 = queryWeight, product of:
                1.0663863 = boost
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.013552261 = queryNorm
              0.4338592 = fieldWeight in 3356, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                3.4708736 = idf(docFreq=3611, maxDocs=42740)
                0.0625 = fieldNorm(doc=3356)
          0.020414183 = weight(abstract_txt:document in 3356) [ClassicSimilarity], result of:
            0.020414183 = score(doc=3356,freq=1.0), product of:
              0.076301046 = queryWeight, product of:
                1.3152165 = boost
                4.280766 = idf(docFreq=1606, maxDocs=42740)
                0.013552261 = queryNorm
              0.26754788 = fieldWeight in 3356, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.280766 = idf(docFreq=1606, maxDocs=42740)
                0.0625 = fieldNorm(doc=3356)
          0.019685702 = weight(abstract_txt:different in 3356) [ClassicSimilarity], result of:
            0.019685702 = score(doc=3356,freq=1.0), product of:
              0.08525245 = queryWeight, product of:
                1.7026724 = boost
                3.694571 = idf(docFreq=2887, maxDocs=42740)
                0.013552261 = queryNorm
              0.23091069 = fieldWeight in 3356, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.694571 = idf(docFreq=2887, maxDocs=42740)
                0.0625 = fieldNorm(doc=3356)
          0.4352102 = weight(abstract_txt:judgment in 3356) [ClassicSimilarity], result of:
            0.4352102 = score(doc=3356,freq=8.0), product of:
              0.3357703 = queryWeight, product of:
                3.3790824 = boost
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.013552261 = queryNorm
              1.2961545 = fieldWeight in 3356, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                7.332157 = idf(docFreq=75, maxDocs=42740)
                0.0625 = fieldNorm(doc=3356)
          0.1624486 = weight(abstract_txt:relevance in 3356) [ClassicSimilarity], result of:
            0.1624486 = score(doc=3356,freq=3.0), product of:
              0.3041257 = queryWeight, product of:
                4.547987 = boost
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.013552261 = queryNorm
              0.5341495 = fieldWeight in 3356, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.934262 = idf(docFreq=835, maxDocs=42740)
                0.0625 = fieldNorm(doc=3356)
        0.2 = coord(5/25)