Document (#20266)

Author

Harter, S.P.
Hert, C.A.

Title

Evaluation of information retrieval systems : approaches, issues, and methods

Source

Annual review of information science and technology. 32(1997), S.3-94

Year

1997

Abstract

State of the art review of information retrieval systems, defined as systems retrieving documents a sopposed to numerical data. Explains the classic Cranfield studies that have served as a standard for retrieval testing since the 1960s and discusses the Cranfield model and its relevance based measures of retrieval effectiveness. Details sosme of the problems with the Cranfield instruments and issues of validity and reliability, generalizability, usefulness and basic concepts. Discusses the evaluation of the Internet search engines in light of the Cranfield model, noting the very real differences between batch systems (Cranfield) and interactive systems (Internet). Because the Internet collection is not fixed, it is impossible to determine recall as a measure of retrieval effectiveness. considers future directions in evaluating information retrieval systems

Theme

Retrievalstudien
Literaturübersicht

Similar documents (author)

1. Hert, C.A.: Exploring a new model for understanding information retrieval interactions (1992) 2.14

   2.1432674 = sum of:
     2.1432674 = product of:
       4.286535 = sum of:
         4.286535 = weight(author_txt:hert in 4521) [ClassicSimilarity], result of:
           4.286535 = score(doc=4521,freq=1.0), product of:
             0.7402546 = queryWeight, product of:
               1.0493017 = boost
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.076143965 = queryNorm
             5.790622 = fieldWeight in 4521, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.625 = fieldNorm(doc=4521)
       0.5 = coord(1/2)
   

2. Hert, C.A.: Information retrieval as situated action (1995) 2.14

   2.1432674 = sum of:
     2.1432674 = product of:
       4.286535 = sum of:
         4.286535 = weight(author_txt:hert in 3386) [ClassicSimilarity], result of:
           4.286535 = score(doc=3386,freq=1.0), product of:
             0.7402546 = queryWeight, product of:
               1.0493017 = boost
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.076143965 = queryNorm
             5.790622 = fieldWeight in 3386, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.625 = fieldNorm(doc=3386)
       0.5 = coord(1/2)
   

3. Hert, C.A.: Information retrieval as situated action (1995) 2.14

   2.1432674 = sum of:
     2.1432674 = product of:
       4.286535 = sum of:
         4.286535 = weight(author_txt:hert in 3893) [ClassicSimilarity], result of:
           4.286535 = score(doc=3893,freq=1.0), product of:
             0.7402546 = queryWeight, product of:
               1.0493017 = boost
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.076143965 = queryNorm
             5.790622 = fieldWeight in 3893, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.625 = fieldNorm(doc=3893)
       0.5 = coord(1/2)
   

4. Hert, C.A.: User goals on an online public access catalog (1996) 2.14

   2.1432674 = sum of:
     2.1432674 = product of:
       4.286535 = sum of:
         4.286535 = weight(author_txt:hert in 4450) [ClassicSimilarity], result of:
           4.286535 = score(doc=4450,freq=1.0), product of:
             0.7402546 = queryWeight, product of:
               1.0493017 = boost
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.076143965 = queryNorm
             5.790622 = fieldWeight in 4450, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.625 = fieldNorm(doc=4450)
       0.5 = coord(1/2)
   

5. Hert, C.A.: Understanding information retrieval interactions : theoretical and practical implications (1997) 2.14

   2.1432674 = sum of:
     2.1432674 = product of:
       4.286535 = sum of:
         4.286535 = weight(author_txt:hert in 5062) [ClassicSimilarity], result of:
           4.286535 = score(doc=5062,freq=1.0), product of:
             0.7402546 = queryWeight, product of:
               1.0493017 = boost
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.076143965 = queryNorm
             5.790622 = fieldWeight in 5062, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.264996 = idf(docFreq=10, maxDocs=42740)
               0.625 = fieldNorm(doc=5062)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Borlund, P.; Ruthven, I.: Introduction to the special issue on evaluating interactive information retrieval systems (2008) 0.26

   0.25902602 = sum of:
     0.25902602 = product of:
       0.647565 = sum of:
         0.035898343 = weight(abstract_txt:reliability in 4020) [ClassicSimilarity], result of:
           0.035898343 = score(doc=4020,freq=1.0), product of:
             0.110701926 = queryWeight, product of:
               1.1074421 = boost
               6.9179583 = idf(docFreq=114, maxDocs=42740)
               0.014449612 = queryNorm
             0.3242793 = fieldWeight in 4020, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9179583 = idf(docFreq=114, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.0046680598 = weight(abstract_txt:information in 4020) [ClassicSimilarity], result of:
           0.0046680598 = score(doc=4020,freq=1.0), product of:
             0.04097984 = queryWeight, product of:
               1.1670493 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.014449612 = queryNorm
             0.11391112 = fieldWeight in 4020, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.0434483 = weight(abstract_txt:instruments in 4020) [ClassicSimilarity], result of:
           0.0434483 = score(doc=4020,freq=1.0), product of:
             0.12572475 = queryWeight, product of:
               1.1801952 = boost
               7.3724313 = idf(docFreq=72, maxDocs=42740)
               0.014449612 = queryNorm
             0.34558272 = fieldWeight in 4020, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3724313 = idf(docFreq=72, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.030094879 = weight(abstract_txt:issues in 4020) [ClassicSimilarity], result of:
           0.030094879 = score(doc=4020,freq=3.0), product of:
             0.08598121 = queryWeight, product of:
               1.3802578 = boost
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.014449612 = queryNorm
             0.35001692 = fieldWeight in 4020, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.05899765 = weight(abstract_txt:evaluation in 4020) [ClassicSimilarity], result of:
           0.05899765 = score(doc=4020,freq=9.0), product of:
             0.09338086 = queryWeight, product of:
               1.4384255 = boost
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.014449612 = queryNorm
             0.63179594 = fieldWeight in 4020, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.029063944 = weight(abstract_txt:effectiveness in 4020) [ClassicSimilarity], result of:
           0.029063944 = score(doc=4020,freq=1.0), product of:
             0.12115795 = queryWeight, product of:
               1.6384546 = boost
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.014449612 = queryNorm
             0.23988473 = fieldWeight in 4020, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.016557716 = weight(abstract_txt:internet in 4020) [ClassicSimilarity], result of:
           0.016557716 = score(doc=4020,freq=1.0), product of:
             0.09531171 = queryWeight, product of:
               1.7798246 = boost
               3.7060635 = idf(docFreq=2854, maxDocs=42740)
               0.014449612 = queryNorm
             0.17372173 = fieldWeight in 4020, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7060635 = idf(docFreq=2854, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.06852816 = weight(abstract_txt:systems in 4020) [ClassicSimilarity], result of:
           0.06852816 = score(doc=4020,freq=7.0), product of:
             0.16182144 = queryWeight, product of:
               3.2797225 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.014449612 = queryNorm
             0.42348012 = fieldWeight in 4020, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.046869185 = weight(abstract_txt:retrieval in 4020) [ClassicSimilarity], result of:
           0.046869185 = score(doc=4020,freq=3.0), product of:
             0.16661234 = queryWeight, product of:
               3.3279183 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.014449612 = queryNorm
             0.2813068 = fieldWeight in 4020, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
         0.31343883 = weight(abstract_txt:cranfield in 4020) [ClassicSimilarity], result of:
           0.31343883 = score(doc=4020,freq=1.0), product of:
             0.80265844 = queryWeight, product of:
               6.667973 = boost
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.014449612 = queryNorm
             0.3905009 = fieldWeight in 4020, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.046875 = fieldNorm(doc=4020)
       0.4 = coord(10/25)
   

2. Cleverdon, C.W.; Mills, J.: ¬The testing of index language devices (1985) 0.18

   0.18424492 = sum of:
     0.18424492 = product of:
       1.1515307 = sum of:
         0.0066016335 = weight(abstract_txt:information in 4644) [ClassicSimilarity], result of:
           0.0066016335 = score(doc=4644,freq=2.0), product of:
             0.04097984 = queryWeight, product of:
               1.1670493 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.014449612 = queryNorm
             0.16109467 = fieldWeight in 4644, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.046875 = fieldNorm(doc=4644)
         0.044862214 = weight(abstract_txt:systems in 4644) [ClassicSimilarity], result of:
           0.044862214 = score(doc=4644,freq=3.0), product of:
             0.16182144 = queryWeight, product of:
               3.2797225 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.014449612 = queryNorm
             0.27723283 = fieldWeight in 4644, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.046875 = fieldNorm(doc=4644)
         0.06050786 = weight(abstract_txt:retrieval in 4644) [ClassicSimilarity], result of:
           0.06050786 = score(doc=4644,freq=5.0), product of:
             0.16661234 = queryWeight, product of:
               3.3279183 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.014449612 = queryNorm
             0.36316553 = fieldWeight in 4644, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.046875 = fieldNorm(doc=4644)
         1.039559 = weight(abstract_txt:cranfield in 4644) [ClassicSimilarity], result of:
           1.039559 = score(doc=4644,freq=11.0), product of:
             0.80265844 = queryWeight, product of:
               6.667973 = boost
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.014449612 = queryNorm
             1.2951449 = fieldWeight in 4644, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.046875 = fieldNorm(doc=4644)
       0.16 = coord(4/25)
   

3. Fugmann, R.: ¬The complementarity of natural and indexing languages (1985) 0.17

   0.17032951 = sum of:
     0.17032951 = product of:
       0.7097063 = sum of:
         0.034375265 = weight(abstract_txt:validity in 4642) [ClassicSimilarity], result of:
           0.034375265 = score(doc=4642,freq=1.0), product of:
             0.10754815 = queryWeight, product of:
               1.0915532 = boost
               6.8187037 = idf(docFreq=126, maxDocs=42740)
               0.014449612 = queryNorm
             0.31962675 = fieldWeight in 4642, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8187037 = idf(docFreq=126, maxDocs=42740)
               0.046875 = fieldNorm(doc=4642)
         0.0066016335 = weight(abstract_txt:information in 4642) [ClassicSimilarity], result of:
           0.0066016335 = score(doc=4642,freq=2.0), product of:
             0.04097984 = queryWeight, product of:
               1.1670493 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.014449612 = queryNorm
             0.16109467 = fieldWeight in 4642, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.046875 = fieldNorm(doc=4642)
         0.045816332 = weight(abstract_txt:1960s in 4642) [ClassicSimilarity], result of:
           0.045816332 = score(doc=4642,freq=1.0), product of:
             0.13025242 = queryWeight, product of:
               1.2012582 = boost
               7.5040073 = idf(docFreq=63, maxDocs=42740)
               0.014449612 = queryNorm
             0.35175034 = fieldWeight in 4642, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5040073 = idf(docFreq=63, maxDocs=42740)
               0.046875 = fieldNorm(doc=4642)
         0.025901213 = weight(abstract_txt:systems in 4642) [ClassicSimilarity], result of:
           0.025901213 = score(doc=4642,freq=1.0), product of:
             0.16182144 = queryWeight, product of:
               3.2797225 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.014449612 = queryNorm
             0.16006045 = fieldWeight in 4642, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.046875 = fieldNorm(doc=4642)
         0.054119874 = weight(abstract_txt:retrieval in 4642) [ClassicSimilarity], result of:
           0.054119874 = score(doc=4642,freq=4.0), product of:
             0.16661234 = queryWeight, product of:
               3.3279183 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.014449612 = queryNorm
             0.3248251 = fieldWeight in 4642, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.046875 = fieldNorm(doc=4642)
         0.542892 = weight(abstract_txt:cranfield in 4642) [ClassicSimilarity], result of:
           0.542892 = score(doc=4642,freq=3.0), product of:
             0.80265844 = queryWeight, product of:
               6.667973 = boost
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.014449612 = queryNorm
             0.6763674 = fieldWeight in 4642, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.046875 = fieldNorm(doc=4642)
       0.24 = coord(6/25)
   

4. Gillman, P.: Text retrieval (1998) 0.15

   0.1533625 = sum of:
     0.1533625 = product of:
       0.95851564 = sum of:
         0.01089214 = weight(abstract_txt:information in 2503) [ClassicSimilarity], result of:
           0.01089214 = score(doc=2503,freq=1.0), product of:
             0.04097984 = queryWeight, product of:
               1.1670493 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.014449612 = queryNorm
             0.26579264 = fieldWeight in 2503, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.109375 = fieldNorm(doc=2503)
         0.10690478 = weight(abstract_txt:1960s in 2503) [ClassicSimilarity], result of:
           0.10690478 = score(doc=2503,freq=1.0), product of:
             0.13025242 = queryWeight, product of:
               1.2012582 = boost
               7.5040073 = idf(docFreq=63, maxDocs=42740)
               0.014449612 = queryNorm
             0.82075083 = fieldWeight in 2503, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5040073 = idf(docFreq=63, maxDocs=42740)
               0.109375 = fieldNorm(doc=2503)
         0.109361425 = weight(abstract_txt:retrieval in 2503) [ClassicSimilarity], result of:
           0.109361425 = score(doc=2503,freq=3.0), product of:
             0.16661234 = queryWeight, product of:
               3.3279183 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.014449612 = queryNorm
             0.6563825 = fieldWeight in 2503, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.109375 = fieldNorm(doc=2503)
         0.7313573 = weight(abstract_txt:cranfield in 2503) [ClassicSimilarity], result of:
           0.7313573 = score(doc=2503,freq=1.0), product of:
             0.80265844 = queryWeight, product of:
               6.667973 = boost
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.014449612 = queryNorm
             0.91116875 = fieldWeight in 2503, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.109375 = fieldNorm(doc=2503)
       0.16 = coord(4/25)
   

5. Voorhees, E.M.: On test collections for adaptive information retrieval (2008) 0.15

   0.1522083 = sum of:
     0.1522083 = product of:
       0.95130193 = sum of:
         0.06850437 = weight(abstract_txt:effectiveness in 4445) [ClassicSimilarity], result of:
           0.06850437 = score(doc=4445,freq=2.0), product of:
             0.12115795 = queryWeight, product of:
               1.6384546 = boost
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.014449612 = queryNorm
             0.5654137 = fieldWeight in 4445, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.078125 = fieldNorm(doc=4445)
         0.043168686 = weight(abstract_txt:systems in 4445) [ClassicSimilarity], result of:
           0.043168686 = score(doc=4445,freq=1.0), product of:
             0.16182144 = queryWeight, product of:
               3.2797225 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.014449612 = queryNorm
             0.2667674 = fieldWeight in 4445, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.078125 = fieldNorm(doc=4445)
         0.10084643 = weight(abstract_txt:retrieval in 4445) [ClassicSimilarity], result of:
           0.10084643 = score(doc=4445,freq=5.0), product of:
             0.16661234 = queryWeight, product of:
               3.3279183 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.014449612 = queryNorm
             0.60527587 = fieldWeight in 4445, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.078125 = fieldNorm(doc=4445)
         0.7387824 = weight(abstract_txt:cranfield in 4445) [ClassicSimilarity], result of:
           0.7387824 = score(doc=4445,freq=2.0), product of:
             0.80265844 = queryWeight, product of:
               6.667973 = boost
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.014449612 = queryNorm
             0.9204194 = fieldWeight in 4445, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.330686 = idf(docFreq=27, maxDocs=42740)
               0.078125 = fieldNorm(doc=4445)
       0.16 = coord(4/25)