Document (#13228)

Author

Huffman, G.D.

Title

Semi-automatic determination of citation relevancy : user evaluation

Source

Information processing and management. 26(1990) no.2, S.295-302

Year

1990

Abstract

Online bibiographic, database searches typically produce hundreds of retrieved citations with only about 20-40% relevant to the search topic and/or problem statement. Significant amounts of time are required to categorize and select the relevant citations. A software system-SORT-AIDS/SABRE-has been developes which ranks the citations in terms of relevance. This paper presents the results of a comprehensive user evaluation of the relevance ranking procedures. Test results show that the software generated distributions approach the ideal distribution-all relevant citations at the beginning of the collection-in 22% of the cases, are 23% better than the random distribution-relevant citations distributed uniformly throughout the dcollection-on average and are poorer than the random distribution in 4% of the cae.

Object

SORT-AID/SABRE

Similar documents (content)

1. Ruthven, I.; Lalmas, M.; Rijsbergen, K. van: Combining and selecting characteristics of information use (2002) 0.16

   0.15837851 = sum of:
     0.15837851 = product of:
       0.5656375 = sum of:
         0.06422002 = weight(abstract_txt:ranks in 5208) [ClassicSimilarity], result of:
           0.06422002 = score(doc=5208,freq=1.0), product of:
             0.1805991 = queryWeight, product of:
               1.1613436 = boost
               7.5860133 = idf(docFreq=60, maxDocs=44218)
               0.020499405 = queryNorm
             0.35559437 = fieldWeight in 5208, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5860133 = idf(docFreq=60, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
         0.024588162 = weight(abstract_txt:than in 5208) [ClassicSimilarity], result of:
           0.024588162 = score(doc=5208,freq=2.0), product of:
             0.095225364 = queryWeight, product of:
               1.1925985 = boost
               3.8950868 = idf(docFreq=2444, maxDocs=44218)
               0.020499405 = queryNorm
             0.2582102 = fieldWeight in 5208, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.8950868 = idf(docFreq=2444, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
         0.11828707 = weight(abstract_txt:poorer in 5208) [ClassicSimilarity], result of:
           0.11828707 = score(doc=5208,freq=1.0), product of:
             0.27136892 = queryWeight, product of:
               1.4235836 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.020499405 = queryNorm
             0.43589026 = fieldWeight in 5208, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
         0.04991185 = weight(abstract_txt:relevance in 5208) [ClassicSimilarity], result of:
           0.04991185 = score(doc=5208,freq=2.0), product of:
             0.15266438 = queryWeight, product of:
               1.5100342 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.020499405 = queryNorm
             0.32693842 = fieldWeight in 5208, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
         0.08200085 = weight(abstract_txt:random in 5208) [ClassicSimilarity], result of:
           0.08200085 = score(doc=5208,freq=1.0), product of:
             0.26780838 = queryWeight, product of:
               2.0 = boost
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.020499405 = queryNorm
             0.30619225 = fieldWeight in 5208, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
         0.10940351 = weight(abstract_txt:distribution in 5208) [ClassicSimilarity], result of:
           0.10940351 = score(doc=5208,freq=2.0), product of:
             0.29488534 = queryWeight, product of:
               2.5703373 = boost
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.020499405 = queryNorm
             0.37100357 = fieldWeight in 5208, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
         0.11722607 = weight(abstract_txt:relevant in 5208) [ClassicSimilarity], result of:
           0.11722607 = score(doc=5208,freq=4.0), product of:
             0.26974377 = queryWeight, product of:
               2.838629 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.020499405 = queryNorm
             0.43458307 = fieldWeight in 5208, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.046875 = fieldNorm(doc=5208)
       0.28 = coord(7/25)
   

2. Bensman, S.J.; Smolinsky, L.J.; Pudovkin, A.I.: Mean citation rate per article in mathematics journals : differences from the scientific model (2010) 0.13

   0.13316298 = sum of:
     0.13316298 = product of:
       0.55484575 = sum of:
         0.07704906 = weight(abstract_txt:distributions in 3595) [ClassicSimilarity], result of:
           0.07704906 = score(doc=3595,freq=2.0), product of:
             0.14604016 = queryWeight, product of:
               1.0443332 = boost
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.020499405 = queryNorm
             0.5275882 = fieldWeight in 3595, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3595)
         0.07492336 = weight(abstract_txt:ranks in 3595) [ClassicSimilarity], result of:
           0.07492336 = score(doc=3595,freq=1.0), product of:
             0.1805991 = queryWeight, product of:
               1.1613436 = boost
               7.5860133 = idf(docFreq=60, maxDocs=44218)
               0.020499405 = queryNorm
             0.4148601 = fieldWeight in 3595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5860133 = idf(docFreq=60, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3595)
         0.020284198 = weight(abstract_txt:than in 3595) [ClassicSimilarity], result of:
           0.020284198 = score(doc=3595,freq=1.0), product of:
             0.095225364 = queryWeight, product of:
               1.1925985 = boost
               3.8950868 = idf(docFreq=2444, maxDocs=44218)
               0.020499405 = queryNorm
             0.21301256 = fieldWeight in 3595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.8950868 = idf(docFreq=2444, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3595)
         0.09566765 = weight(abstract_txt:random in 3595) [ClassicSimilarity], result of:
           0.09566765 = score(doc=3595,freq=1.0), product of:
             0.26780838 = queryWeight, product of:
               2.0 = boost
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.020499405 = queryNorm
             0.3572243 = fieldWeight in 3595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3595)
         0.15632328 = weight(abstract_txt:distribution in 3595) [ClassicSimilarity], result of:
           0.15632328 = score(doc=3595,freq=3.0), product of:
             0.29488534 = queryWeight, product of:
               2.5703373 = boost
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.020499405 = queryNorm
             0.5301155 = fieldWeight in 3595, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3595)
         0.13059819 = weight(abstract_txt:citations in 3595) [ClassicSimilarity], result of:
           0.13059819 = score(doc=3595,freq=1.0), product of:
             0.44728622 = queryWeight, product of:
               4.0867743 = boost
               5.339045 = idf(docFreq=576, maxDocs=44218)
               0.020499405 = queryNorm
             0.291979 = fieldWeight in 3595, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.339045 = idf(docFreq=576, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3595)
       0.24 = coord(6/25)
   

3. Joss, M.W.; Wszola, S.: ¬The engines that can : text search and retrieval software, their strategies, and vendors (1996) 0.12

   0.12350986 = sum of:
     0.12350986 = product of:
       0.44110665 = sum of:
         0.06469765 = weight(abstract_txt:amounts in 5123) [ClassicSimilarity], result of:
           0.06469765 = score(doc=5123,freq=1.0), product of:
             0.14981954 = queryWeight, product of:
               1.05776 = boost
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.020499405 = queryNorm
             0.43183723 = fieldWeight in 5123, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
         0.01656695 = weight(abstract_txt:results in 5123) [ClassicSimilarity], result of:
           0.01656695 = score(doc=5123,freq=1.0), product of:
             0.07611691 = queryWeight, product of:
               1.0662487 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.020499405 = queryNorm
             0.21765138 = fieldWeight in 5123, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
         0.07881909 = weight(abstract_txt:sort in 5123) [ClassicSimilarity], result of:
           0.07881909 = score(doc=5123,freq=1.0), product of:
             0.1708954 = queryWeight, product of:
               1.1297129 = boost
               7.3793993 = idf(docFreq=74, maxDocs=44218)
               0.020499405 = queryNorm
             0.46121246 = fieldWeight in 5123, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3793993 = idf(docFreq=74, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
         0.023181941 = weight(abstract_txt:than in 5123) [ClassicSimilarity], result of:
           0.023181941 = score(doc=5123,freq=1.0), product of:
             0.095225364 = queryWeight, product of:
               1.1925985 = boost
               3.8950868 = idf(docFreq=2444, maxDocs=44218)
               0.020499405 = queryNorm
             0.24344292 = fieldWeight in 5123, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.8950868 = idf(docFreq=2444, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
         0.056161407 = weight(abstract_txt:software in 5123) [ClassicSimilarity], result of:
           0.056161407 = score(doc=5123,freq=3.0), product of:
             0.11909796 = queryWeight, product of:
               1.3337367 = boost
               4.3560514 = idf(docFreq=1541, maxDocs=44218)
               0.020499405 = queryNorm
             0.4715564 = fieldWeight in 5123, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.3560514 = idf(docFreq=1541, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
         0.047057346 = weight(abstract_txt:relevance in 5123) [ClassicSimilarity], result of:
           0.047057346 = score(doc=5123,freq=1.0), product of:
             0.15266438 = queryWeight, product of:
               1.5100342 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.020499405 = queryNorm
             0.3082405 = fieldWeight in 5123, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
         0.15462229 = weight(abstract_txt:random in 5123) [ClassicSimilarity], result of:
           0.15462229 = score(doc=5123,freq=2.0), product of:
             0.26780838 = queryWeight, product of:
               2.0 = boost
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.020499405 = queryNorm
             0.57736164 = fieldWeight in 5123, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.0625 = fieldNorm(doc=5123)
       0.28 = coord(7/25)
   

4. Leydesdorff, L.; Bornmann, L.; Mutz, R.; Opthof, T.: Turning the tables on citation analysis one more time : principles for comparing sets of documents (2011) 0.12

   0.11720135 = sum of:
     0.11720135 = product of:
       0.58600676 = sum of:
         0.088056065 = weight(abstract_txt:distributions in 4485) [ClassicSimilarity], result of:
           0.088056065 = score(doc=4485,freq=2.0), product of:
             0.14604016 = queryWeight, product of:
               1.0443332 = boost
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.020499405 = queryNorm
             0.6029579 = fieldWeight in 4485, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.0625 = fieldNorm(doc=4485)
         0.14830978 = weight(abstract_txt:ranks in 4485) [ClassicSimilarity], result of:
           0.14830978 = score(doc=4485,freq=3.0), product of:
             0.1805991 = queryWeight, product of:
               1.1613436 = boost
               7.5860133 = idf(docFreq=60, maxDocs=44218)
               0.020499405 = queryNorm
             0.82121 = fieldWeight in 4485, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.5860133 = idf(docFreq=60, maxDocs=44218)
               0.0625 = fieldNorm(doc=4485)
         0.035415724 = weight(abstract_txt:evaluation in 4485) [ClassicSimilarity], result of:
           0.035415724 = score(doc=4485,freq=1.0), product of:
             0.12631363 = queryWeight, product of:
               1.3735454 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.020499405 = queryNorm
             0.2803793 = fieldWeight in 4485, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=4485)
         0.10314662 = weight(abstract_txt:distribution in 4485) [ClassicSimilarity], result of:
           0.10314662 = score(doc=4485,freq=1.0), product of:
             0.29488534 = queryWeight, product of:
               2.5703373 = boost
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.020499405 = queryNorm
             0.3497855 = fieldWeight in 4485, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.0625 = fieldNorm(doc=4485)
         0.21107855 = weight(abstract_txt:citations in 4485) [ClassicSimilarity], result of:
           0.21107855 = score(doc=4485,freq=2.0), product of:
             0.44728622 = queryWeight, product of:
               4.0867743 = boost
               5.339045 = idf(docFreq=576, maxDocs=44218)
               0.020499405 = queryNorm
             0.47190937 = fieldWeight in 4485, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.339045 = idf(docFreq=576, maxDocs=44218)
               0.0625 = fieldNorm(doc=4485)
       0.2 = coord(5/25)
   

5. Cai, F.; Wang, S.; Rijke, M.de: Behavior-based personalization in web search (2017) 0.12

   0.11508278 = sum of:
     0.11508278 = product of:
       0.4110099 = sum of:
         0.062265042 = weight(abstract_txt:distributions in 3527) [ClassicSimilarity], result of:
           0.062265042 = score(doc=3527,freq=1.0), product of:
             0.14604016 = queryWeight, product of:
               1.0443332 = boost
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.020499405 = queryNorm
             0.42635563 = fieldWeight in 3527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
         0.01656695 = weight(abstract_txt:results in 3527) [ClassicSimilarity], result of:
           0.01656695 = score(doc=3527,freq=1.0), product of:
             0.07611691 = queryWeight, product of:
               1.0662487 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.020499405 = queryNorm
             0.21765138 = fieldWeight in 3527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
         0.033959135 = weight(abstract_txt:user in 3527) [ClassicSimilarity], result of:
           0.033959135 = score(doc=3527,freq=3.0), product of:
             0.08516283 = queryWeight, product of:
               1.1278282 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.020499405 = queryNorm
             0.39875534 = fieldWeight in 3527, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
         0.035415724 = weight(abstract_txt:evaluation in 3527) [ClassicSimilarity], result of:
           0.035415724 = score(doc=3527,freq=1.0), product of:
             0.12631363 = queryWeight, product of:
               1.3735454 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.020499405 = queryNorm
             0.2803793 = fieldWeight in 3527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
         0.081505716 = weight(abstract_txt:relevance in 3527) [ClassicSimilarity], result of:
           0.081505716 = score(doc=3527,freq=3.0), product of:
             0.15266438 = queryWeight, product of:
               1.5100342 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.020499405 = queryNorm
             0.5338882 = fieldWeight in 3527, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
         0.10314662 = weight(abstract_txt:distribution in 3527) [ClassicSimilarity], result of:
           0.10314662 = score(doc=3527,freq=1.0), product of:
             0.29488534 = queryWeight, product of:
               2.5703373 = boost
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.020499405 = queryNorm
             0.3497855 = fieldWeight in 3527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.596568 = idf(docFreq=445, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
         0.07815072 = weight(abstract_txt:relevant in 3527) [ClassicSimilarity], result of:
           0.07815072 = score(doc=3527,freq=1.0), product of:
             0.26974377 = queryWeight, product of:
               2.838629 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.020499405 = queryNorm
             0.28972206 = fieldWeight in 3527, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0625 = fieldNorm(doc=3527)
       0.28 = coord(7/25)