Document (#8322)

Author

Dyer, H.

Title

Beyond relevance, precision and recall : the 'worth' of information retrieval results

Source

Journal of document and text management. 2(1994) no.1, S.35-47

Year

1994

Abstract

The existing tools of relevance, recall and precision have their place in the evaluation of information retrieval systems, but do not tell the whole story. They cannot, for example, make allowance for the enquirer already knowing of the existence of a document or the subsequent availability of items. Utility theory goes part way towards meeting the need by assessing the likely usefulness of documents. Arguing that information is not information until it is used, the paper suggests an additional approach to information system evaluation - the 'use-value' of information

Theme

Information

Similar documents (author)

1. Dyer, S.: ¬The dark side of the moon : introducing IT to the school library (1993) 6.19

   6.190705 = sum of:
     6.190705 = weight(author_txt:dyer in 5537) [ClassicSimilarity], result of:
       6.190705 = fieldWeight in 5537, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.625 = fieldNorm(doc=5537)
   

2. Dyer, H.; Morris, A.: Human aspects of library automation (1990) 4.95

   4.952564 = sum of:
     4.952564 = weight(author_txt:dyer in 2178) [ClassicSimilarity], result of:
       4.952564 = fieldWeight in 2178, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.5 = fieldNorm(doc=2178)
   

3. Dyer, H.; Bouchet, M.-L.: ¬A comparison between the perceived value of information retrieved via end-user searching of CD-ROMs and mediated online searching (1995) 4.33

   4.333493 = sum of:
     4.333493 = weight(author_txt:dyer in 2417) [ClassicSimilarity], result of:
       4.333493 = fieldWeight in 2417, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.4375 = fieldNorm(doc=2417)
   

4. Dyer, H. u. A. Gunson (Comp.): ¬A directory of library and information retrieval software for microcomputers (1990) 3.71

   3.7144227 = sum of:
     3.7144227 = weight(author_txt:dyer in 816) [ClassicSimilarity], result of:
       3.7144227 = fieldWeight in 816, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.905128 = idf(docFreq=5, maxDocs=44218)
         0.375 = fieldNorm(doc=816)
   

Similar documents (content)

1. Lesk, M.E.; Salton, G.: Relevance assements and retrieval system evaluation (1969) 0.24

   0.23694438 = sum of:
     0.23694438 = product of:
       0.8462299 = sum of:
         0.080373384 = weight(abstract_txt:cannot in 4151) [ClassicSimilarity], result of:
           0.080373384 = score(doc=4151,freq=1.0), product of:
             0.14368136 = queryWeight, product of:
               1.0369649 = boost
               5.9667873 = idf(docFreq=307, maxDocs=44218)
               0.023221796 = queryNorm
             0.5593863 = fieldWeight in 4151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9667873 = idf(docFreq=307, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
         0.031756837 = weight(abstract_txt:retrieval in 4151) [ClassicSimilarity], result of:
           0.031756837 = score(doc=4151,freq=1.0), product of:
             0.09747511 = queryWeight, product of:
               1.2078854 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.023221796 = queryNorm
             0.3257943 = fieldWeight in 4151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
         0.068315364 = weight(abstract_txt:evaluation in 4151) [ClassicSimilarity], result of:
           0.068315364 = score(doc=4151,freq=1.0), product of:
             0.16243559 = queryWeight, product of:
               1.5592632 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.023221796 = queryNorm
             0.42056894 = fieldWeight in 4151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
         0.09077153 = weight(abstract_txt:relevance in 4151) [ClassicSimilarity], result of:
           0.09077153 = score(doc=4151,freq=1.0), product of:
             0.19632189 = queryWeight, product of:
               1.7142067 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023221796 = queryNorm
             0.46236074 = fieldWeight in 4151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
         0.2552651 = weight(abstract_txt:precision in 4151) [ClassicSimilarity], result of:
           0.2552651 = score(doc=4151,freq=4.0), product of:
             0.24640141 = queryWeight, product of:
               1.9204386 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.023221796 = queryNorm
             1.0359725 = fieldWeight in 4151, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
         0.28753722 = weight(abstract_txt:recall in 4151) [ClassicSimilarity], result of:
           0.28753722 = score(doc=4151,freq=4.0), product of:
             0.26675442 = queryWeight, product of:
               1.9981802 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.023221796 = queryNorm
             1.0779098 = fieldWeight in 4151, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
         0.03221049 = weight(abstract_txt:information in 4151) [ClassicSimilarity], result of:
           0.03221049 = score(doc=4151,freq=1.0), product of:
             0.14191912 = queryWeight, product of:
               2.5244102 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.023221796 = queryNorm
             0.22696373 = fieldWeight in 4151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.09375 = fieldNorm(doc=4151)
       0.28 = coord(7/25)
   

2. Wallis, P.; Thom, J.A.: Relevance judgements for assessing recall (1996) 0.19

   0.19103836 = sum of:
     0.19103836 = product of:
       0.79599315 = sum of:
         0.18548852 = weight(abstract_txt:assessing in 4949) [ClassicSimilarity], result of:
           0.18548852 = score(doc=4949,freq=2.0), product of:
             0.17970593 = queryWeight, product of:
               1.1596984 = boost
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.023221796 = queryNorm
             1.032178 = fieldWeight in 4949, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.109375 = fieldNorm(doc=4949)
         0.037049644 = weight(abstract_txt:retrieval in 4949) [ClassicSimilarity], result of:
           0.037049644 = score(doc=4949,freq=1.0), product of:
             0.09747511 = queryWeight, product of:
               1.2078854 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.023221796 = queryNorm
             0.38009337 = fieldWeight in 4949, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.109375 = fieldNorm(doc=4949)
         0.14976539 = weight(abstract_txt:relevance in 4949) [ClassicSimilarity], result of:
           0.14976539 = score(doc=4949,freq=2.0), product of:
             0.19632189 = queryWeight, product of:
               1.7142067 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023221796 = queryNorm
             0.7628563 = fieldWeight in 4949, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.109375 = fieldNorm(doc=4949)
         0.14890464 = weight(abstract_txt:precision in 4949) [ClassicSimilarity], result of:
           0.14890464 = score(doc=4949,freq=1.0), product of:
             0.24640141 = queryWeight, product of:
               1.9204386 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.023221796 = queryNorm
             0.6043173 = fieldWeight in 4949, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.109375 = fieldNorm(doc=4949)
         0.2372061 = weight(abstract_txt:recall in 4949) [ClassicSimilarity], result of:
           0.2372061 = score(doc=4949,freq=2.0), product of:
             0.26675442 = queryWeight, product of:
               1.9981802 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.023221796 = queryNorm
             0.88923025 = fieldWeight in 4949, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.109375 = fieldNorm(doc=4949)
         0.037578907 = weight(abstract_txt:information in 4949) [ClassicSimilarity], result of:
           0.037578907 = score(doc=4949,freq=1.0), product of:
             0.14191912 = queryWeight, product of:
               2.5244102 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.023221796 = queryNorm
             0.264791 = fieldWeight in 4949, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.109375 = fieldNorm(doc=4949)
       0.24 = coord(6/25)
   

3. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 0.16

   0.15786831 = sum of:
     0.15786831 = product of:
       0.65778464 = sum of:
         0.063513674 = weight(abstract_txt:retrieval in 3308) [ClassicSimilarity], result of:
           0.063513674 = score(doc=3308,freq=4.0), product of:
             0.09747511 = queryWeight, product of:
               1.2078854 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.023221796 = queryNorm
             0.6515886 = fieldWeight in 3308, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.09375 = fieldNorm(doc=3308)
         0.068315364 = weight(abstract_txt:evaluation in 3308) [ClassicSimilarity], result of:
           0.068315364 = score(doc=3308,freq=1.0), product of:
             0.16243559 = queryWeight, product of:
               1.5592632 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.023221796 = queryNorm
             0.42056894 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=3308)
         0.22234395 = weight(abstract_txt:relevance in 3308) [ClassicSimilarity], result of:
           0.22234395 = score(doc=3308,freq=6.0), product of:
             0.19632189 = queryWeight, product of:
               1.7142067 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023221796 = queryNorm
             1.132548 = fieldWeight in 3308, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.09375 = fieldNorm(doc=3308)
         0.12763254 = weight(abstract_txt:precision in 3308) [ClassicSimilarity], result of:
           0.12763254 = score(doc=3308,freq=1.0), product of:
             0.24640141 = queryWeight, product of:
               1.9204386 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.023221796 = queryNorm
             0.51798624 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.09375 = fieldNorm(doc=3308)
         0.14376861 = weight(abstract_txt:recall in 3308) [ClassicSimilarity], result of:
           0.14376861 = score(doc=3308,freq=1.0), product of:
             0.26675442 = queryWeight, product of:
               1.9981802 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.023221796 = queryNorm
             0.5389549 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.09375 = fieldNorm(doc=3308)
         0.03221049 = weight(abstract_txt:information in 3308) [ClassicSimilarity], result of:
           0.03221049 = score(doc=3308,freq=1.0), product of:
             0.14191912 = queryWeight, product of:
               2.5244102 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.023221796 = queryNorm
             0.22696373 = fieldWeight in 3308, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.09375 = fieldNorm(doc=3308)
       0.24 = coord(6/25)
   

4. Spink, A.; Greisdorf, H.: Regions and levels : Measuring and mapping users' relevance judgements (2001) 0.15

   0.15351692 = sum of:
     0.15351692 = product of:
       0.5482747 = sum of:
         0.0560921 = weight(abstract_txt:usefulness in 5586) [ClassicSimilarity], result of:
           0.0560921 = score(doc=5586,freq=1.0), product of:
             0.14813381 = queryWeight, product of:
               1.0529093 = boost
               6.0585327 = idf(docFreq=280, maxDocs=44218)
               0.023221796 = queryNorm
             0.3786583 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0585327 = idf(docFreq=280, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.06472235 = weight(abstract_txt:utility in 5586) [ClassicSimilarity], result of:
           0.06472235 = score(doc=5586,freq=1.0), product of:
             0.1629631 = queryWeight, product of:
               1.1043545 = boost
               6.3545527 = idf(docFreq=208, maxDocs=44218)
               0.023221796 = queryNorm
             0.39715955 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.3545527 = idf(docFreq=208, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.021171225 = weight(abstract_txt:retrieval in 5586) [ClassicSimilarity], result of:
           0.021171225 = score(doc=5586,freq=1.0), product of:
             0.09747511 = queryWeight, product of:
               1.2078854 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.023221796 = queryNorm
             0.21719621 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.06440835 = weight(abstract_txt:evaluation in 5586) [ClassicSimilarity], result of:
           0.06440835 = score(doc=5586,freq=2.0), product of:
             0.16243559 = queryWeight, product of:
               1.5592632 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.023221796 = queryNorm
             0.3965162 = fieldWeight in 5586, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.22642401 = weight(abstract_txt:relevance in 5586) [ClassicSimilarity], result of:
           0.22642401 = score(doc=5586,freq=14.0), product of:
             0.19632189 = queryWeight, product of:
               1.7142067 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023221796 = queryNorm
             1.1533304 = fieldWeight in 5586, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.085088365 = weight(abstract_txt:precision in 5586) [ClassicSimilarity], result of:
           0.085088365 = score(doc=5586,freq=1.0), product of:
             0.24640141 = queryWeight, product of:
               1.9204386 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.023221796 = queryNorm
             0.34532416 = fieldWeight in 5586, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
         0.030368343 = weight(abstract_txt:information in 5586) [ClassicSimilarity], result of:
           0.030368343 = score(doc=5586,freq=2.0), product of:
             0.14191912 = queryWeight, product of:
               2.5244102 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.023221796 = queryNorm
             0.21398345 = fieldWeight in 5586, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.0625 = fieldNorm(doc=5586)
       0.28 = coord(7/25)
   

5. Vakkari, P.; Huuskonen, S.: Search effort degrades search output but improves task outcome (2012) 0.14

   0.14360805 = sum of:
     0.14360805 = product of:
       0.51288587 = sum of:
         0.0560921 = weight(abstract_txt:usefulness in 46) [ClassicSimilarity], result of:
           0.0560921 = score(doc=46,freq=1.0), product of:
             0.14813381 = queryWeight, product of:
               1.0529093 = boost
               6.0585327 = idf(docFreq=280, maxDocs=44218)
               0.023221796 = queryNorm
             0.3786583 = fieldWeight in 46, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0585327 = idf(docFreq=280, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
         0.074948676 = weight(abstract_txt:assessing in 46) [ClassicSimilarity], result of:
           0.074948676 = score(doc=46,freq=1.0), product of:
             0.17970593 = queryWeight, product of:
               1.1596984 = boost
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.023221796 = queryNorm
             0.4170629 = fieldWeight in 46, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
         0.029940631 = weight(abstract_txt:retrieval in 46) [ClassicSimilarity], result of:
           0.029940631 = score(doc=46,freq=2.0), product of:
             0.09747511 = queryWeight, product of:
               1.2078854 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.023221796 = queryNorm
             0.3071618 = fieldWeight in 46, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
         0.06440835 = weight(abstract_txt:evaluation in 46) [ClassicSimilarity], result of:
           0.06440835 = score(doc=46,freq=2.0), product of:
             0.16243559 = queryWeight, product of:
               1.5592632 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.023221796 = queryNorm
             0.3965162 = fieldWeight in 46, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
         0.17017673 = weight(abstract_txt:precision in 46) [ClassicSimilarity], result of:
           0.17017673 = score(doc=46,freq=4.0), product of:
             0.24640141 = queryWeight, product of:
               1.9204386 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.023221796 = queryNorm
             0.6906483 = fieldWeight in 46, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
         0.09584574 = weight(abstract_txt:recall in 46) [ClassicSimilarity], result of:
           0.09584574 = score(doc=46,freq=1.0), product of:
             0.26675442 = queryWeight, product of:
               1.9981802 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.023221796 = queryNorm
             0.35930327 = fieldWeight in 46, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
         0.021473661 = weight(abstract_txt:information in 46) [ClassicSimilarity], result of:
           0.021473661 = score(doc=46,freq=1.0), product of:
             0.14191912 = queryWeight, product of:
               2.5244102 = boost
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.023221796 = queryNorm
             0.15130915 = fieldWeight in 46, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4209464 = idf(docFreq=10677, maxDocs=44218)
               0.0625 = fieldNorm(doc=46)
       0.28 = coord(7/25)