Document (#28650)

Author

Lancaster, F.W.

Title

Evaluating the performance of a large computerized information system

Source

Theory of subject analysis: a sourcebook. Ed.: L.M. Chan, et al

Imprint

Littleton, CO : Libraries Unlimited

Year

1985

Pages

S.326-339

Abstract

F. W. Lancaster is known for his writing an the state of the art in librarylinformation science. His skill in identifying significant contributions and synthesizing literature in fields as diverse as online systems, vocabulary control, measurement and evaluation, and the paperless society have earned him esteem as a chronicler of information science. Equally deserving of repute is his own contribution to research in the discipline-his evaluation of the MEDLARS operating system. The MEDLARS study is notable for several reasons. It was the first large-scale application of retrieval experiment methodology to the evaluation of an actual operating system. As such, problems had to be faced that do not arise in laboratory-like conditions. One example is the problem of recall: how to determine, for a very large and dynamic database, the number of documents relevant to a given search request. By solving this problem and others attendant upon transferring an experimental methodology to the real world, Lancaster created a constructive procedure that could be used to improve the design and functioning of retrieval systems. The MEDLARS study is notable also for its contribution to our understanding of what constitutes a good index language and good indexing. The ideal retrieval system would be one that retrieves all and only relevant documents. The failures that occur in real operating systems, when a relevant document is not retrieved (a recall failure) or an irrelevant document is retrieved (a precision failure), can be analysed to assess the impact of various factors an the performance of the system. This is exactly what Lancaster did. He found both the MEDLARS indexing and the McSH index language to be significant factors affecting retrieval performance. The indexing, primarily because it was insufficiently exhaustive, explained a large number of recall failures. The index language, largely because of its insufficient specificity, accounted for a large number of precision failures. The purpose of identifying factors responsible for a system's failures is ultimately to improve the system. Unlike many user studies, the MEDLARS evaluation yielded recommendations that were eventually implemented.* Indexing exhaustivity was increased and the McSH index language was enriched with more specific terms and a larger entry vocabulary.

Footnote

Nachdruck des Originalartikels mit Kommentierung durch die Herausgeber
Original in: Journal of the American Medical Association 207(1969) S.114-120.

Theme

Retrievalstudien

Object

Medlars

Similar documents (author)

1. Lancaster, F.W.: Vocabulary control in information retrieval systems (1977) 4.93

   4.931391 = sum of:
     4.931391 = weight(author_txt:lancaster in 1774) [ClassicSimilarity], result of:
       4.931391 = fieldWeight in 1774, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.890225 = idf(docFreq=44, maxDocs=44218)
         0.625 = fieldNorm(doc=1774)
   

2. Lancaster, F.W.: Toward paperless information systems (1978) 4.93

   4.931391 = sum of:
     4.931391 = weight(author_txt:lancaster in 3356) [ClassicSimilarity], result of:
       4.931391 = fieldWeight in 3356, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.890225 = idf(docFreq=44, maxDocs=44218)
         0.625 = fieldNorm(doc=3356)
   

3. Lancaster, F.W.: Libraries and librarians in an age of electronics (1982) 4.93

   4.931391 = sum of:
     4.931391 = weight(author_txt:lancaster in 6184) [ClassicSimilarity], result of:
       4.931391 = fieldWeight in 6184, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.890225 = idf(docFreq=44, maxDocs=44218)
         0.625 = fieldNorm(doc=6184)
   

4. Lancaster, F.W.: Librarians, technology and mediocrity (1993) 4.93

   4.931391 = sum of:
     4.931391 = weight(author_txt:lancaster in 7006) [ClassicSimilarity], result of:
       4.931391 = fieldWeight in 7006, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.890225 = idf(docFreq=44, maxDocs=44218)
         0.625 = fieldNorm(doc=7006)
   

5. Lancaster, F.W.: Information retrieval systems : characteristics, testing and evaluation (1979) 4.93

   4.931391 = sum of:
     4.931391 = weight(author_txt:lancaster in 7270) [ClassicSimilarity], result of:
       4.931391 = fieldWeight in 7270, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         7.890225 = idf(docFreq=44, maxDocs=44218)
         0.625 = fieldNorm(doc=7270)
   

Similar documents (content)

1. Tonta, Y.: Analysis of search failures in document retrieval systems : a review (1992) 0.33

   0.32784888 = sum of:
     0.32784888 = product of:
       1.0245278 = sum of:
         0.05467805 = weight(abstract_txt:precision in 4611) [ClassicSimilarity], result of:
           0.05467805 = score(doc=4611,freq=1.0), product of:
             0.09047904 = queryWeight, product of:
               1.0311468 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.015881099 = queryNorm
             0.6043173 = fieldWeight in 4611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.010781239 = weight(abstract_txt:that in 4611) [ClassicSimilarity], result of:
           0.010781239 = score(doc=4611,freq=1.0), product of:
             0.041600488 = queryWeight, product of:
               1.1055177 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015881099 = queryNorm
             0.25916135 = fieldWeight in 4611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.048275255 = weight(abstract_txt:performance in 4611) [ClassicSimilarity], result of:
           0.048275255 = score(doc=4611,freq=1.0), product of:
             0.095320456 = queryWeight, product of:
               1.2962393 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.015881099 = queryNorm
             0.5064522 = fieldWeight in 4611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.06664913 = weight(abstract_txt:retrieval in 4611) [ClassicSimilarity], result of:
           0.06664913 = score(doc=4611,freq=6.0), product of:
             0.07158607 = queryWeight, product of:
               1.2971069 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015881099 = queryNorm
             0.93103486 = fieldWeight in 4611, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.1118948 = weight(abstract_txt:failure in 4611) [ClassicSimilarity], result of:
           0.1118948 = score(doc=4611,freq=1.0), product of:
             0.14584087 = queryWeight, product of:
               1.3091401 = boost
               7.014756 = idf(docFreq=107, maxDocs=44218)
               0.015881099 = queryNorm
             0.767239 = fieldWeight in 4611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.014756 = idf(docFreq=107, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.09238615 = weight(abstract_txt:recall in 4611) [ClassicSimilarity], result of:
           0.09238615 = score(doc=4611,freq=1.0), product of:
             0.14692904 = queryWeight, product of:
               1.6093332 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.015881099 = queryNorm
             0.6287807 = fieldWeight in 4611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.03729712 = weight(abstract_txt:system in 4611) [ClassicSimilarity], result of:
           0.03729712 = score(doc=4611,freq=1.0), product of:
             0.10111833 = queryWeight, product of:
               1.8880873 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.015881099 = queryNorm
             0.36884624 = fieldWeight in 4611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
         0.60256606 = weight(abstract_txt:failures in 4611) [ClassicSimilarity], result of:
           0.60256606 = score(doc=4611,freq=4.0), product of:
             0.35563058 = queryWeight, product of:
               2.8910863 = boost
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.015881099 = queryNorm
             1.6943595 = fieldWeight in 4611, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.109375 = fieldNorm(doc=4611)
       0.32 = coord(8/25)
   

2. Cleverdon, C.W.; Mills, J.: ¬The testing of index language devices (1985) 0.30

   0.29931295 = sum of:
     0.29931295 = product of:
       0.5344874 = sum of:
         0.04058792 = weight(abstract_txt:precision in 3643) [ClassicSimilarity], result of:
           0.04058792 = score(doc=3643,freq=3.0), product of:
             0.09047904 = queryWeight, product of:
               1.0311468 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.015881099 = queryNorm
             0.4485892 = fieldWeight in 3643, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.033637475 = weight(abstract_txt:good in 3643) [ClassicSimilarity], result of:
           0.033637475 = score(doc=3643,freq=2.0), product of:
             0.091382444 = queryWeight, product of:
               1.0362818 = boost
               5.5527015 = idf(docFreq=465, maxDocs=44218)
               0.015881099 = queryNorm
             0.36809558 = fieldWeight in 3643, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5527015 = idf(docFreq=465, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.036538046 = weight(abstract_txt:retrieved in 3643) [ClassicSimilarity], result of:
           0.036538046 = score(doc=3643,freq=2.0), product of:
             0.096563 = queryWeight, product of:
               1.0652508 = boost
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.015881099 = queryNorm
             0.37838557 = fieldWeight in 3643, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.013861592 = weight(abstract_txt:that in 3643) [ClassicSimilarity], result of:
           0.013861592 = score(doc=3643,freq=9.0), product of:
             0.041600488 = queryWeight, product of:
               1.1055177 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015881099 = queryNorm
             0.33320743 = fieldWeight in 3643, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.020689394 = weight(abstract_txt:performance in 3643) [ClassicSimilarity], result of:
           0.020689394 = score(doc=3643,freq=1.0), product of:
             0.095320456 = queryWeight, product of:
               1.2962393 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.015881099 = queryNorm
             0.21705094 = fieldWeight in 3643, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.026075164 = weight(abstract_txt:retrieval in 3643) [ClassicSimilarity], result of:
           0.026075164 = score(doc=3643,freq=5.0), product of:
             0.07158607 = queryWeight, product of:
               1.2971069 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015881099 = queryNorm
             0.3642491 = fieldWeight in 3643, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.029356632 = weight(abstract_txt:relevant in 3643) [ClassicSimilarity], result of:
           0.029356632 = score(doc=3643,freq=2.0), product of:
             0.0955319 = queryWeight, product of:
               1.2976762 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.015881099 = queryNorm
             0.30729663 = fieldWeight in 3643, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.025446171 = weight(abstract_txt:factors in 3643) [ClassicSimilarity], result of:
           0.025446171 = score(doc=3643,freq=1.0), product of:
             0.10942147 = queryWeight, product of:
               1.3888118 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.015881099 = queryNorm
             0.2325519 = fieldWeight in 3643, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.06824385 = weight(abstract_txt:notable in 3643) [ClassicSimilarity], result of:
           0.06824385 = score(doc=3643,freq=1.0), product of:
             0.18451549 = queryWeight, product of:
               1.4725258 = boost
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.015881099 = queryNorm
             0.3698543 = fieldWeight in 3643, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.028742181 = weight(abstract_txt:language in 3643) [ClassicSimilarity], result of:
           0.028742181 = score(doc=3643,freq=2.0), product of:
             0.10367411 = queryWeight, product of:
               1.5609775 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015881099 = queryNorm
             0.27723587 = fieldWeight in 3643, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.06857893 = weight(abstract_txt:recall in 3643) [ClassicSimilarity], result of:
           0.06857893 = score(doc=3643,freq=3.0), product of:
             0.14692904 = queryWeight, product of:
               1.6093332 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.015881099 = queryNorm
             0.46674863 = fieldWeight in 3643, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.060494866 = weight(abstract_txt:indexing in 3643) [ClassicSimilarity], result of:
           0.060494866 = score(doc=3643,freq=7.0), product of:
             0.11214507 = queryWeight, product of:
               1.6234974 = boost
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.015881099 = queryNorm
             0.539434 = fieldWeight in 3643, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.05154408 = weight(abstract_txt:index in 3643) [ClassicSimilarity], result of:
           0.05154408 = score(doc=3643,freq=3.0), product of:
             0.13368395 = queryWeight, product of:
               1.7725607 = boost
               4.74895 = idf(docFreq=1040, maxDocs=44218)
               0.015881099 = queryNorm
             0.3855667 = fieldWeight in 3643, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.74895 = idf(docFreq=1040, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
         0.030691076 = weight(abstract_txt:large in 3643) [ClassicSimilarity], result of:
           0.030691076 = score(doc=3643,freq=1.0), product of:
             0.14699817 = queryWeight, product of:
               2.0781288 = boost
               4.454089 = idf(docFreq=1397, maxDocs=44218)
               0.015881099 = queryNorm
             0.20878543 = fieldWeight in 3643, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.454089 = idf(docFreq=1397, maxDocs=44218)
               0.046875 = fieldNorm(doc=3643)
       0.56 = coord(14/25)
   

3. Dumais, S.T.: Latent semantic analysis (2003) 0.27

   0.2694737 = sum of:
     0.2694737 = product of:
       0.48120302 = sum of:
         0.028497942 = weight(abstract_txt:vocabulary in 2462) [ClassicSimilarity], result of:
           0.028497942 = score(doc=2462,freq=4.0), product of:
             0.085095584 = queryWeight, product of:
               5.358293 = idf(docFreq=565, maxDocs=44218)
               0.015881099 = queryNorm
             0.33489332 = fieldWeight in 2462, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.358293 = idf(docFreq=565, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.015856856 = weight(abstract_txt:good in 2462) [ClassicSimilarity], result of:
           0.015856856 = score(doc=2462,freq=1.0), product of:
             0.091382444 = queryWeight, product of:
               1.0362818 = boost
               5.5527015 = idf(docFreq=465, maxDocs=44218)
               0.015881099 = queryNorm
             0.17352192 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5527015 = idf(docFreq=465, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.024358697 = weight(abstract_txt:retrieved in 2462) [ClassicSimilarity], result of:
           0.024358697 = score(doc=2462,freq=2.0), product of:
             0.096563 = queryWeight, product of:
               1.0652508 = boost
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.015881099 = queryNorm
             0.25225705 = fieldWeight in 2462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.009241061 = weight(abstract_txt:that in 2462) [ClassicSimilarity], result of:
           0.009241061 = score(doc=2462,freq=9.0), product of:
             0.041600488 = queryWeight, product of:
               1.1055177 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015881099 = queryNorm
             0.22213829 = fieldWeight in 2462, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.009805752 = weight(abstract_txt:number in 2462) [ClassicSimilarity], result of:
           0.009805752 = score(doc=2462,freq=1.0), product of:
             0.07592807 = queryWeight, product of:
               1.1568934 = boost
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.015881099 = queryNorm
             0.12914528 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.023322335 = weight(abstract_txt:retrieval in 2462) [ClassicSimilarity], result of:
           0.023322335 = score(doc=2462,freq=9.0), product of:
             0.07158607 = queryWeight, product of:
               1.2971069 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015881099 = queryNorm
             0.3257943 = fieldWeight in 2462, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.023969589 = weight(abstract_txt:relevant in 2462) [ClassicSimilarity], result of:
           0.023969589 = score(doc=2462,freq=3.0), product of:
             0.0955319 = queryWeight, product of:
               1.2976762 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.015881099 = queryNorm
             0.25090665 = fieldWeight in 2462, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.016964113 = weight(abstract_txt:factors in 2462) [ClassicSimilarity], result of:
           0.016964113 = score(doc=2462,freq=1.0), product of:
             0.10942147 = queryWeight, product of:
               1.3888118 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.015881099 = queryNorm
             0.1550346 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.013549193 = weight(abstract_txt:language in 2462) [ClassicSimilarity], result of:
           0.013549193 = score(doc=2462,freq=1.0), product of:
             0.10367411 = queryWeight, product of:
               1.5609775 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015881099 = queryNorm
             0.13069023 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.021557247 = weight(abstract_txt:indexing in 2462) [ClassicSimilarity], result of:
           0.021557247 = score(doc=2462,freq=2.0), product of:
             0.11214507 = queryWeight, product of:
               1.6234974 = boost
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.015881099 = queryNorm
             0.19222642 = fieldWeight in 2462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.3495874 = idf(docFreq=1551, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.02805704 = weight(abstract_txt:index in 2462) [ClassicSimilarity], result of:
           0.02805704 = score(doc=2462,freq=2.0), product of:
             0.13368395 = queryWeight, product of:
               1.7725607 = boost
               4.74895 = idf(docFreq=1040, maxDocs=44218)
               0.015881099 = queryNorm
             0.20987591 = fieldWeight in 2462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.74895 = idf(docFreq=1040, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.045751553 = weight(abstract_txt:large in 2462) [ClassicSimilarity], result of:
           0.045751553 = score(doc=2462,freq=5.0), product of:
             0.14699817 = queryWeight, product of:
               2.0781288 = boost
               4.454089 = idf(docFreq=1397, maxDocs=44218)
               0.015881099 = queryNorm
             0.31123894 = fieldWeight in 2462, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.454089 = idf(docFreq=1397, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.13419077 = weight(abstract_txt:lancaster in 2462) [ClassicSimilarity], result of:
           0.13419077 = score(doc=2462,freq=2.0), product of:
             0.34478912 = queryWeight, product of:
               2.465295 = boost
               8.806516 = idf(docFreq=17, maxDocs=44218)
               0.015881099 = queryNorm
             0.38919666 = fieldWeight in 2462, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.806516 = idf(docFreq=17, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
         0.086080864 = weight(abstract_txt:failures in 2462) [ClassicSimilarity], result of:
           0.086080864 = score(doc=2462,freq=1.0), product of:
             0.35563058 = queryWeight, product of:
               2.8910863 = boost
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.015881099 = queryNorm
             0.24205136 = fieldWeight in 2462, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.03125 = fieldNorm(doc=2462)
       0.56 = coord(14/25)
   

4. Croft, W.B.: Effective retrieval based on combining evidence from the corpus and users (1995) 0.24

   0.23956546 = sum of:
     0.23956546 = product of:
       0.7486421 = sum of:
         0.062489197 = weight(abstract_txt:precision in 4489) [ClassicSimilarity], result of:
           0.062489197 = score(doc=4489,freq=1.0), product of:
             0.09047904 = queryWeight, product of:
               1.0311468 = boost
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.015881099 = queryNorm
             0.6906483 = fieldWeight in 4489, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5251865 = idf(docFreq=478, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.012321415 = weight(abstract_txt:that in 4489) [ClassicSimilarity], result of:
           0.012321415 = score(doc=4489,freq=1.0), product of:
             0.041600488 = queryWeight, product of:
               1.1055177 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015881099 = queryNorm
             0.2961844 = fieldWeight in 4489, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.039223008 = weight(abstract_txt:number in 4489) [ClassicSimilarity], result of:
           0.039223008 = score(doc=4489,freq=1.0), product of:
             0.07592807 = queryWeight, product of:
               1.1568934 = boost
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.015881099 = queryNorm
             0.5165811 = fieldWeight in 4489, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.043977015 = weight(abstract_txt:retrieval in 4489) [ClassicSimilarity], result of:
           0.043977015 = score(doc=4489,freq=2.0), product of:
             0.07158607 = queryWeight, product of:
               1.2971069 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015881099 = queryNorm
             0.6143236 = fieldWeight in 4489, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.10558417 = weight(abstract_txt:recall in 4489) [ClassicSimilarity], result of:
           0.10558417 = score(doc=4489,freq=1.0), product of:
             0.14692904 = queryWeight, product of:
               1.6093332 = boost
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.015881099 = queryNorm
             0.71860653 = fieldWeight in 4489, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7488523 = idf(docFreq=382, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.066894725 = weight(abstract_txt:evaluation in 4489) [ClassicSimilarity], result of:
           0.066894725 = score(doc=4489,freq=1.0), product of:
             0.119293265 = queryWeight, product of:
               1.6744394 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015881099 = queryNorm
             0.5607586 = fieldWeight in 4489, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.073829144 = weight(abstract_txt:system in 4489) [ClassicSimilarity], result of:
           0.073829144 = score(doc=4489,freq=3.0), product of:
             0.10111833 = queryWeight, product of:
               1.8880873 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.015881099 = queryNorm
             0.7301262 = fieldWeight in 4489, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
         0.34432346 = weight(abstract_txt:failures in 4489) [ClassicSimilarity], result of:
           0.34432346 = score(doc=4489,freq=1.0), product of:
             0.35563058 = queryWeight, product of:
               2.8910863 = boost
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.015881099 = queryNorm
             0.96820545 = fieldWeight in 4489, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.125 = fieldNorm(doc=4489)
       0.32 = coord(8/25)
   

5. Saracevic, T.: Effects of inconsistent relevance judgments on information retrieval test results : a historical perspective (2008) 0.24

   0.23522167 = sum of:
     0.23522167 = product of:
       0.7350677 = sum of:
         0.0061607077 = weight(abstract_txt:that in 5585) [ClassicSimilarity], result of:
           0.0061607077 = score(doc=5585,freq=1.0), product of:
             0.041600488 = queryWeight, product of:
               1.1055177 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.015881099 = queryNorm
             0.1480922 = fieldWeight in 5585, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.019611504 = weight(abstract_txt:number in 5585) [ClassicSimilarity], result of:
           0.019611504 = score(doc=5585,freq=1.0), product of:
             0.07592807 = queryWeight, product of:
               1.1568934 = boost
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.015881099 = queryNorm
             0.25829056 = fieldWeight in 5585, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.132649 = idf(docFreq=1927, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.02758586 = weight(abstract_txt:performance in 5585) [ClassicSimilarity], result of:
           0.02758586 = score(doc=5585,freq=1.0), product of:
             0.095320456 = queryWeight, product of:
               1.2962393 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.015881099 = queryNorm
             0.28940126 = fieldWeight in 5585, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.026930314 = weight(abstract_txt:retrieval in 5585) [ClassicSimilarity], result of:
           0.026930314 = score(doc=5585,freq=3.0), product of:
             0.07158607 = queryWeight, product of:
               1.2971069 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.015881099 = queryNorm
             0.37619486 = fieldWeight in 5585, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.0276777 = weight(abstract_txt:relevant in 5585) [ClassicSimilarity], result of:
           0.0276777 = score(doc=5585,freq=1.0), product of:
             0.0955319 = queryWeight, product of:
               1.2976762 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.015881099 = queryNorm
             0.28972206 = fieldWeight in 5585, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.057932526 = weight(abstract_txt:evaluation in 5585) [ClassicSimilarity], result of:
           0.057932526 = score(doc=5585,freq=3.0), product of:
             0.119293265 = queryWeight, product of:
               1.6744394 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015881099 = queryNorm
             0.48563117 = fieldWeight in 5585, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.17216173 = weight(abstract_txt:failures in 5585) [ClassicSimilarity], result of:
           0.17216173 = score(doc=5585,freq=1.0), product of:
             0.35563058 = queryWeight, product of:
               2.8910863 = boost
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.015881099 = queryNorm
             0.48410273 = fieldWeight in 5585, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
         0.39700744 = weight(abstract_txt:medlars in 5585) [ClassicSimilarity], result of:
           0.39700744 = score(doc=5585,freq=1.0), product of:
             0.66866785 = queryWeight, product of:
               4.4322243 = boost
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.015881099 = queryNorm
             0.5937289 = fieldWeight in 5585, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.499662 = idf(docFreq=8, maxDocs=44218)
               0.0625 = fieldNorm(doc=5585)
       0.32 = coord(8/25)