Document (#20265)

Author

Harter, S.P.
Hert, C.A.

Title

Evaluation of information retrieval systems : approaches, issues, and methods

Source

Annual review of information science and technology. 32(1997), S.3-94

Year

1997

Abstract

State of the art review of information retrieval systems, defined as systems retrieving documents a sopposed to numerical data. Explains the classic Cranfield studies that have served as a standard for retrieval testing since the 1960s and discusses the Cranfield model and its relevance based measures of retrieval effectiveness. Details sosme of the problems with the Cranfield instruments and issues of validity and reliability, generalizability, usefulness and basic concepts. Discusses the evaluation of the Internet search engines in light of the Cranfield model, noting the very real differences between batch systems (Cranfield) and interactive systems (Internet). Because the Internet collection is not fixed, it is impossible to determine recall as a measure of retrieval effectiveness. considers future directions in evaluating information retrieval systems

Theme

Retrievalstudien
Literaturübersicht

Similar documents (author)

1. Hert, C.A.: Exploring a new model for understanding information retrieval interactions (1992) 2.15

   2.1507812 = sum of:
     2.1507812 = product of:
       4.3015623 = sum of:
         4.3015623 = weight(author_txt:hert in 4521) [ClassicSimilarity], result of:
           4.3015623 = score(doc=4521,freq=1.0), product of:
             0.74013394 = queryWeight, product of:
               1.0491126 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.0758669 = queryNorm
             5.81187 = fieldWeight in 4521, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.625 = fieldNorm(doc=4521)
       0.5 = coord(1/2)
   

2. Hert, C.A.: Information retrieval as situated action (1995) 2.15

   2.1507812 = sum of:
     2.1507812 = product of:
       4.3015623 = sum of:
         4.3015623 = weight(author_txt:hert in 3317) [ClassicSimilarity], result of:
           4.3015623 = score(doc=3317,freq=1.0), product of:
             0.74013394 = queryWeight, product of:
               1.0491126 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.0758669 = queryNorm
             5.81187 = fieldWeight in 3317, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.625 = fieldNorm(doc=3317)
       0.5 = coord(1/2)
   

3. Hert, C.A.: Information retrieval as situated action (1995) 2.15

   2.1507812 = sum of:
     2.1507812 = product of:
       4.3015623 = sum of:
         4.3015623 = weight(author_txt:hert in 3824) [ClassicSimilarity], result of:
           4.3015623 = score(doc=3824,freq=1.0), product of:
             0.74013394 = queryWeight, product of:
               1.0491126 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.0758669 = queryNorm
             5.81187 = fieldWeight in 3824, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.625 = fieldNorm(doc=3824)
       0.5 = coord(1/2)
   

4. Hert, C.A.: User goals on an online public access catalog (1996) 2.15

   2.1507812 = sum of:
     2.1507812 = product of:
       4.3015623 = sum of:
         4.3015623 = weight(author_txt:hert in 4381) [ClassicSimilarity], result of:
           4.3015623 = score(doc=4381,freq=1.0), product of:
             0.74013394 = queryWeight, product of:
               1.0491126 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.0758669 = queryNorm
             5.81187 = fieldWeight in 4381, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.625 = fieldNorm(doc=4381)
       0.5 = coord(1/2)
   

5. Hert, C.A.: Understanding information retrieval interactions : theoretical and practical implications (1997) 2.15

   2.1507812 = sum of:
     2.1507812 = product of:
       4.3015623 = sum of:
         4.3015623 = weight(author_txt:hert in 4061) [ClassicSimilarity], result of:
           4.3015623 = score(doc=4061,freq=1.0), product of:
             0.74013394 = queryWeight, product of:
               1.0491126 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.0758669 = queryNorm
             5.81187 = fieldWeight in 4061, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.625 = fieldNorm(doc=4061)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Borlund, P.; Ruthven, I.: Introduction to the special issue on evaluating interactive information retrieval systems (2008) 0.26

   0.25882858 = sum of:
     0.25882858 = product of:
       0.6470714 = sum of:
         0.035356905 = weight(abstract_txt:reliability in 2019) [ClassicSimilarity], result of:
           0.035356905 = score(doc=2019,freq=1.0), product of:
             0.11007057 = queryWeight, product of:
               1.0979136 = boost
               6.8527 = idf(docFreq=126, maxDocs=44218)
               0.014629899 = queryNorm
             0.32122034 = fieldWeight in 2019, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8527 = idf(docFreq=126, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.004676983 = weight(abstract_txt:information in 2019) [ClassicSimilarity], result of:
           0.004676983 = score(doc=2019,freq=1.0), product of:
             0.041213486 = queryWeight, product of:
               1.1636245 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.014629899 = queryNorm
             0.113481864 = fieldWeight in 2019, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.043914843 = weight(abstract_txt:instruments in 2019) [ClassicSimilarity], result of:
           0.043914843 = score(doc=2019,freq=1.0), product of:
             0.12718306 = queryWeight, product of:
               1.1801771 = boost
               7.3661537 = idf(docFreq=75, maxDocs=44218)
               0.014629899 = queryNorm
             0.34528846 = fieldWeight in 2019, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3661537 = idf(docFreq=75, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.030298712 = weight(abstract_txt:issues in 2019) [ClassicSimilarity], result of:
           0.030298712 = score(doc=2019,freq=3.0), product of:
             0.08675092 = queryWeight, product of:
               1.3784295 = boost
               4.3017797 = idf(docFreq=1627, maxDocs=44218)
               0.014629899 = queryNorm
             0.349261 = fieldWeight in 2019, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.3017797 = idf(docFreq=1627, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.05951659 = weight(abstract_txt:evaluation in 2019) [ClassicSimilarity], result of:
           0.05951659 = score(doc=2019,freq=9.0), product of:
             0.09434298 = queryWeight, product of:
               1.4374818 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.014629899 = queryNorm
             0.6308534 = fieldWeight in 2019, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.029121596 = weight(abstract_txt:effectiveness in 2019) [ClassicSimilarity], result of:
           0.029121596 = score(doc=2019,freq=1.0), product of:
             0.12185458 = queryWeight, product of:
               1.6336856 = boost
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.014629899 = queryNorm
             0.23898648 = fieldWeight in 2019, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.01706682 = weight(abstract_txt:internet in 2019) [ClassicSimilarity], result of:
           0.01706682 = score(doc=2019,freq=1.0), product of:
             0.0976856 = queryWeight, product of:
               1.7914655 = boost
               3.7271836 = idf(docFreq=2891, maxDocs=44218)
               0.014629899 = queryNorm
             0.17471173 = fieldWeight in 2019, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7271836 = idf(docFreq=2891, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.069273934 = weight(abstract_txt:systems in 2019) [ClassicSimilarity], result of:
           0.069273934 = score(doc=2019,freq=7.0), product of:
             0.16371408 = queryWeight, product of:
               3.279829 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.014629899 = queryNorm
             0.42313975 = fieldWeight in 2019, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.047920085 = weight(abstract_txt:retrieval in 2019) [ClassicSimilarity], result of:
           0.047920085 = score(doc=2019,freq=3.0), product of:
             0.16984135 = queryWeight, product of:
               3.3406417 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.014629899 = queryNorm
             0.28214616 = fieldWeight in 2019, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
         0.30992496 = weight(abstract_txt:cranfield in 2019) [ClassicSimilarity], result of:
           0.30992496 = score(doc=2019,freq=1.0), product of:
             0.80017096 = queryWeight, product of:
               6.6192527 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.014629899 = queryNorm
             0.38732344 = fieldWeight in 2019, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.046875 = fieldNorm(doc=2019)
       0.4 = coord(10/25)
   

2. Cleverdon, C.W.; Mills, J.: ¬The testing of index language devices (1985) 0.18

   0.18267748 = sum of:
     0.18267748 = product of:
       1.1417342 = sum of:
         0.0066142534 = weight(abstract_txt:information in 3643) [ClassicSimilarity], result of:
           0.0066142534 = score(doc=3643,freq=2.0), product of:
             0.041213486 = queryWeight, product of:
               1.1636245 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.014629899 = queryNorm
             0.16048759 = fieldWeight in 3643, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.045350436 = weight(abstract_txt:systems in 3643) [ClassicSimilarity], result of:
           0.045350436 = score(doc=3643,freq=3.0), product of:
             0.16371408 = queryWeight, product of:
               3.279829 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.014629899 = queryNorm
             0.27701 = fieldWeight in 3643, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.06186456 = weight(abstract_txt:retrieval in 3643) [ClassicSimilarity], result of:
           0.06186456 = score(doc=3643,freq=5.0), product of:
             0.16984135 = queryWeight, product of:
               3.3406417 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.014629899 = queryNorm
             0.3642491 = fieldWeight in 3643, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         1.027905 = weight(abstract_txt:cranfield in 3643) [ClassicSimilarity], result of:
           1.027905 = score(doc=3643,freq=11.0), product of:
             0.80017096 = queryWeight, product of:
               6.6192527 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.014629899 = queryNorm
             1.2846067 = fieldWeight in 3643, product of:
               3.3166249 = tf(freq=11.0), with freq of:
                 11.0 = termFreq=11.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
       0.16 = coord(4/25)
   

3. Fugmann, R.: ¬The complementarity of natural and indexing languages (1985) 0.17

   0.16930848 = sum of:
     0.16930848 = product of:
       0.705452 = sum of:
         0.03430766 = weight(abstract_txt:validity in 3641) [ClassicSimilarity], result of:
           0.03430766 = score(doc=3641,freq=1.0), product of:
             0.10788204 = queryWeight, product of:
               1.0869439 = boost
               6.784232 = idf(docFreq=135, maxDocs=44218)
               0.014629899 = queryNorm
             0.31801087 = fieldWeight in 3641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.784232 = idf(docFreq=135, maxDocs=44218)
               0.046875 = fieldNorm(doc=3641)
         0.0066142534 = weight(abstract_txt:information in 3641) [ClassicSimilarity], result of:
           0.0066142534 = score(doc=3641,freq=2.0), product of:
             0.041213486 = queryWeight, product of:
               1.1636245 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.014629899 = queryNorm
             0.16048759 = fieldWeight in 3641, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.046875 = fieldNorm(doc=3641)
         0.046207894 = weight(abstract_txt:1960s in 3641) [ClassicSimilarity], result of:
           0.046207894 = score(doc=3641,freq=1.0), product of:
             0.13157272 = queryWeight, product of:
               1.2003709 = boost
               7.4921947 = idf(docFreq=66, maxDocs=44218)
               0.014629899 = queryNorm
             0.35119662 = fieldWeight in 3641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4921947 = idf(docFreq=66, maxDocs=44218)
               0.046875 = fieldNorm(doc=3641)
         0.026183087 = weight(abstract_txt:systems in 3641) [ClassicSimilarity], result of:
           0.026183087 = score(doc=3641,freq=1.0), product of:
             0.16371408 = queryWeight, product of:
               3.279829 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.014629899 = queryNorm
             0.1599318 = fieldWeight in 3641, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.046875 = fieldNorm(doc=3641)
         0.055333346 = weight(abstract_txt:retrieval in 3641) [ClassicSimilarity], result of:
           0.055333346 = score(doc=3641,freq=4.0), product of:
             0.16984135 = queryWeight, product of:
               3.3406417 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.014629899 = queryNorm
             0.3257943 = fieldWeight in 3641, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.046875 = fieldNorm(doc=3641)
         0.5368058 = weight(abstract_txt:cranfield in 3641) [ClassicSimilarity], result of:
           0.5368058 = score(doc=3641,freq=3.0), product of:
             0.80017096 = queryWeight, product of:
               6.6192527 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.014629899 = queryNorm
             0.67086387 = fieldWeight in 3641, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.046875 = fieldNorm(doc=3641)
       0.24 = coord(6/25)
   

4. Gillman, P.: Text retrieval (1998) 0.15

   0.15259251 = sum of:
     0.15259251 = product of:
       0.9537032 = sum of:
         0.01091296 = weight(abstract_txt:information in 1502) [ClassicSimilarity], result of:
           0.01091296 = score(doc=1502,freq=1.0), product of:
             0.041213486 = queryWeight, product of:
               1.1636245 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.014629899 = queryNorm
             0.264791 = fieldWeight in 1502, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.109375 = fieldNorm(doc=1502)
         0.107818425 = weight(abstract_txt:1960s in 1502) [ClassicSimilarity], result of:
           0.107818425 = score(doc=1502,freq=1.0), product of:
             0.13157272 = queryWeight, product of:
               1.2003709 = boost
               7.4921947 = idf(docFreq=66, maxDocs=44218)
               0.014629899 = queryNorm
             0.8194588 = fieldWeight in 1502, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4921947 = idf(docFreq=66, maxDocs=44218)
               0.109375 = fieldNorm(doc=1502)
         0.11181352 = weight(abstract_txt:retrieval in 1502) [ClassicSimilarity], result of:
           0.11181352 = score(doc=1502,freq=3.0), product of:
             0.16984135 = queryWeight, product of:
               3.3406417 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.014629899 = queryNorm
             0.658341 = fieldWeight in 1502, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.109375 = fieldNorm(doc=1502)
         0.7231583 = weight(abstract_txt:cranfield in 1502) [ClassicSimilarity], result of:
           0.7231583 = score(doc=1502,freq=1.0), product of:
             0.80017096 = queryWeight, product of:
               6.6192527 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.014629899 = queryNorm
             0.9037547 = fieldWeight in 1502, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.109375 = fieldNorm(doc=1502)
       0.16 = coord(4/25)
   

5. Voorhees, E.M.: On test collections for adaptive information retrieval (2008) 0.15

   0.15134184 = sum of:
     0.15134184 = product of:
       0.9458865 = sum of:
         0.06864027 = weight(abstract_txt:effectiveness in 2444) [ClassicSimilarity], result of:
           0.06864027 = score(doc=2444,freq=2.0), product of:
             0.12185458 = queryWeight, product of:
               1.6336856 = boost
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.014629899 = queryNorm
             0.56329656 = fieldWeight in 2444, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.078125 = fieldNorm(doc=2444)
         0.043638475 = weight(abstract_txt:systems in 2444) [ClassicSimilarity], result of:
           0.043638475 = score(doc=2444,freq=1.0), product of:
             0.16371408 = queryWeight, product of:
               3.279829 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.014629899 = queryNorm
             0.26655298 = fieldWeight in 2444, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=2444)
         0.10310761 = weight(abstract_txt:retrieval in 2444) [ClassicSimilarity], result of:
           0.10310761 = score(doc=2444,freq=5.0), product of:
             0.16984135 = queryWeight, product of:
               3.3406417 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.014629899 = queryNorm
             0.6070819 = fieldWeight in 2444, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.078125 = fieldNorm(doc=2444)
         0.73050016 = weight(abstract_txt:cranfield in 2444) [ClassicSimilarity], result of:
           0.73050016 = score(doc=2444,freq=2.0), product of:
             0.80017096 = queryWeight, product of:
               6.6192527 = boost
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.014629899 = queryNorm
             0.91293013 = fieldWeight in 2444, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.2629 = idf(docFreq=30, maxDocs=44218)
               0.078125 = fieldNorm(doc=2444)
       0.16 = coord(4/25)