Document (#24980)

Author

Peters, C.
Braschler, M.

Title

Cross-language system evaluation : the CLEF campaigns

Source

Journal of the American Society for Information Science and technology. 52(2001) no.12, S.1067-1072

Year

2001

Abstract

The goals of the CLEF (Cross-Language Evaluation Forum) series of evaluation campaigns for information retrieval systems operating on European languages are described. The difficulties of organizing an activity which aims at an objective evaluation of systems running on and over a number' of different languages are examined. The discussion includes an analysis of the first results and proposals for possible developments in the future.

Theme

Multilinguale Probleme

Similar documents (author)

1. Peters, C.; Braschler, M.; Clough, P.: Multilingual information retrieval : from research to practice (2012) 4.65

   4.6517563 = sum of:
     4.6517563 = sum of:
       1.4550463 = weight(author_txt:peters in 361) [ClassicSimilarity], result of:
         1.4550463 = score(doc=361,freq=1.0), product of:
           0.509246 = queryWeight, product of:
             7.61935 = idf(docFreq=58, maxDocs=44218)
             0.06683589 = queryNorm
           2.8572562 = fieldWeight in 361, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.61935 = idf(docFreq=58, maxDocs=44218)
             0.375 = fieldNorm(doc=361)
       3.19671 = weight(author_txt:braschler in 361) [ClassicSimilarity], result of:
         3.19671 = score(doc=361,freq=1.0), product of:
           0.860621 = queryWeight, product of:
             1.2999964 = boost
             9.905128 = idf(docFreq=5, maxDocs=44218)
             0.06683589 = queryNorm
           3.7144227 = fieldWeight in 361, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.905128 = idf(docFreq=5, maxDocs=44218)
             0.375 = fieldNorm(doc=361)
   

2. Agosti, M.; Braschler, M.; Ferro, N.; Peters, C.; Siebinga, S.: Roadmap for multiLingual information access in the European Library (2007) 3.88

   3.8764637 = sum of:
     3.8764637 = sum of:
       1.2125385 = weight(author_txt:peters in 2431) [ClassicSimilarity], result of:
         1.2125385 = score(doc=2431,freq=1.0), product of:
           0.509246 = queryWeight, product of:
             7.61935 = idf(docFreq=58, maxDocs=44218)
             0.06683589 = queryNorm
           2.3810468 = fieldWeight in 2431, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.61935 = idf(docFreq=58, maxDocs=44218)
             0.3125 = fieldNorm(doc=2431)
       2.6639252 = weight(author_txt:braschler in 2431) [ClassicSimilarity], result of:
         2.6639252 = score(doc=2431,freq=1.0), product of:
           0.860621 = queryWeight, product of:
             1.2999964 = boost
             9.905128 = idf(docFreq=5, maxDocs=44218)
             0.06683589 = queryNorm
           3.0953524 = fieldWeight in 2431, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.905128 = idf(docFreq=5, maxDocs=44218)
             0.3125 = fieldNorm(doc=2431)
   

3. Peters, C.M.: CD-ROM: its potential in libraries (1986) 1.21

   1.2125385 = sum of:
     1.2125385 = product of:
       2.425077 = sum of:
         2.425077 = weight(author_txt:peters in 535) [ClassicSimilarity], result of:
           2.425077 = score(doc=535,freq=1.0), product of:
             0.509246 = queryWeight, product of:
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.06683589 = queryNorm
             4.7620935 = fieldWeight in 535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.625 = fieldNorm(doc=535)
       0.5 = coord(1/2)
   

4. Peters, T.A.: When smart people fail : an analysis of the transaction log of an online public access catalog (1989) 1.21

   1.2125385 = sum of:
     1.2125385 = product of:
       2.425077 = sum of:
         2.425077 = weight(author_txt:peters in 2283) [ClassicSimilarity], result of:
           2.425077 = score(doc=2283,freq=1.0), product of:
             0.509246 = queryWeight, product of:
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.06683589 = queryNorm
             4.7620935 = fieldWeight in 2283, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.625 = fieldNorm(doc=2283)
       0.5 = coord(1/2)
   

5. Peters, C.M.: CD-ROM and optical technology : the user interface (1988) 1.21

   1.2125385 = sum of:
     1.2125385 = product of:
       2.425077 = sum of:
         2.425077 = weight(author_txt:peters in 4013) [ClassicSimilarity], result of:
           2.425077 = score(doc=4013,freq=1.0), product of:
             0.509246 = queryWeight, product of:
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.06683589 = queryNorm
             4.7620935 = fieldWeight in 4013, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.625 = fieldNorm(doc=4013)
       0.5 = coord(1/2)
   

Similar documents (content)

1. García Cumbreras, M.A.; Perea-Ortega, J.M.; García Vega, M.; Ureña López, L.A.: Information retrieval with geographical references : relevant documents filtering vs. query expansion (2009) 0.20

   0.19657615 = sum of:
     0.19657615 = product of:
       0.8190673 = sum of:
         0.025586512 = weight(abstract_txt:systems in 4222) [ClassicSimilarity], result of:
           0.025586512 = score(doc=4222,freq=1.0), product of:
             0.07999195 = queryWeight, product of:
               1.4753067 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.015891703 = queryNorm
             0.3198636 = fieldWeight in 4222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.09375 = fieldNorm(doc=4222)
         0.106248 = weight(abstract_txt:forum in 4222) [ClassicSimilarity], result of:
           0.106248 = score(doc=4222,freq=1.0), product of:
             0.16402476 = queryWeight, product of:
               1.4938221 = boost
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.015891703 = queryNorm
             0.64775586 = fieldWeight in 4222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.09375 = fieldNorm(doc=4222)
         0.047120497 = weight(abstract_txt:language in 4222) [ClassicSimilarity], result of:
           0.047120497 = score(doc=4222,freq=1.0), product of:
             0.12018367 = queryWeight, product of:
               1.8083476 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015891703 = queryNorm
             0.3920707 = fieldWeight in 4222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.09375 = fieldNorm(doc=4222)
         0.1131988 = weight(abstract_txt:cross in 4222) [ClassicSimilarity], result of:
           0.1131988 = score(doc=4222,freq=1.0), product of:
             0.21557586 = queryWeight, product of:
               2.421917 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.015891703 = queryNorm
             0.52509964 = fieldWeight in 4222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.09375 = fieldNorm(doc=4222)
         0.41059244 = weight(abstract_txt:clef in 4222) [ClassicSimilarity], result of:
           0.41059244 = score(doc=4222,freq=1.0), product of:
             0.5089161 = queryWeight, product of:
               3.7211938 = boost
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.015891703 = queryNorm
             0.8067979 = fieldWeight in 4222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.09375 = fieldNorm(doc=4222)
         0.116321035 = weight(abstract_txt:evaluation in 4222) [ClassicSimilarity], result of:
           0.116321035 = score(doc=4222,freq=1.0), product of:
             0.27658018 = queryWeight, product of:
               3.87958 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015891703 = queryNorm
             0.42056894 = fieldWeight in 4222, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=4222)
       0.24 = coord(6/25)
   

2. Kluck, M.; Mandl, T.; Womser-Hacker, C.: Cross-Language Evaluation Forum (CLEF) : Europäische Initiative zur Bewertung sprachübergreifender Retrievalverfahren (2002) 0.19

   0.19271074 = sum of:
     0.19271074 = product of:
       0.96355367 = sum of:
         0.106248 = weight(abstract_txt:forum in 266) [ClassicSimilarity], result of:
           0.106248 = score(doc=266,freq=1.0), product of:
             0.16402476 = queryWeight, product of:
               1.4938221 = boost
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.015891703 = queryNorm
             0.64775586 = fieldWeight in 266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.09375 = fieldNorm(doc=266)
         0.047120497 = weight(abstract_txt:language in 266) [ClassicSimilarity], result of:
           0.047120497 = score(doc=266,freq=1.0), product of:
             0.12018367 = queryWeight, product of:
               1.8083476 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015891703 = queryNorm
             0.3920707 = fieldWeight in 266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.09375 = fieldNorm(doc=266)
         0.1131988 = weight(abstract_txt:cross in 266) [ClassicSimilarity], result of:
           0.1131988 = score(doc=266,freq=1.0), product of:
             0.21557586 = queryWeight, product of:
               2.421917 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.015891703 = queryNorm
             0.52509964 = fieldWeight in 266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.09375 = fieldNorm(doc=266)
         0.58066535 = weight(abstract_txt:clef in 266) [ClassicSimilarity], result of:
           0.58066535 = score(doc=266,freq=2.0), product of:
             0.5089161 = queryWeight, product of:
               3.7211938 = boost
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.015891703 = queryNorm
             1.1409845 = fieldWeight in 266, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.09375 = fieldNorm(doc=266)
         0.116321035 = weight(abstract_txt:evaluation in 266) [ClassicSimilarity], result of:
           0.116321035 = score(doc=266,freq=1.0), product of:
             0.27658018 = queryWeight, product of:
               3.87958 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015891703 = queryNorm
             0.42056894 = fieldWeight in 266, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=266)
       0.2 = coord(5/25)
   

3. Scheufen, J.-H.: RECOIN: Modell offener Schnittstellen für Information-Retrieval-Systeme und -Komponenten (2006) 0.19

   0.18514551 = sum of:
     0.18514551 = product of:
       0.92572755 = sum of:
         0.12395599 = weight(abstract_txt:forum in 4901) [ClassicSimilarity], result of:
           0.12395599 = score(doc=4901,freq=1.0), product of:
             0.16402476 = queryWeight, product of:
               1.4938221 = boost
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.015891703 = queryNorm
             0.75571513 = fieldWeight in 4901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.109375 = fieldNorm(doc=4901)
         0.05497391 = weight(abstract_txt:language in 4901) [ClassicSimilarity], result of:
           0.05497391 = score(doc=4901,freq=1.0), product of:
             0.12018367 = queryWeight, product of:
               1.8083476 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015891703 = queryNorm
             0.45741582 = fieldWeight in 4901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.109375 = fieldNorm(doc=4901)
         0.13206527 = weight(abstract_txt:cross in 4901) [ClassicSimilarity], result of:
           0.13206527 = score(doc=4901,freq=1.0), product of:
             0.21557586 = queryWeight, product of:
               2.421917 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.015891703 = queryNorm
             0.61261624 = fieldWeight in 4901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.109375 = fieldNorm(doc=4901)
         0.47902453 = weight(abstract_txt:clef in 4901) [ClassicSimilarity], result of:
           0.47902453 = score(doc=4901,freq=1.0), product of:
             0.5089161 = queryWeight, product of:
               3.7211938 = boost
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.015891703 = queryNorm
             0.9412643 = fieldWeight in 4901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.109375 = fieldNorm(doc=4901)
         0.13570787 = weight(abstract_txt:evaluation in 4901) [ClassicSimilarity], result of:
           0.13570787 = score(doc=4901,freq=1.0), product of:
             0.27658018 = queryWeight, product of:
               3.87958 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015891703 = queryNorm
             0.49066377 = fieldWeight in 4901, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.109375 = fieldNorm(doc=4901)
       0.2 = coord(5/25)
   

4. Effektive Information Retrieval Verfahren in Theorie und Praxis : ausgewählte und erweiterte Beiträge des Vierten Hildesheimer Evaluierungs- und Retrievalworkshop (HIER 2005), Hildesheim, 20.7.2005 (2006) 0.16

   0.15869616 = sum of:
     0.15869616 = product of:
       0.79348075 = sum of:
         0.106248 = weight(abstract_txt:forum in 5973) [ClassicSimilarity], result of:
           0.106248 = score(doc=5973,freq=1.0), product of:
             0.16402476 = queryWeight, product of:
               1.4938221 = boost
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.015891703 = queryNorm
             0.64775586 = fieldWeight in 5973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9093957 = idf(docFreq=119, maxDocs=44218)
               0.09375 = fieldNorm(doc=5973)
         0.047120497 = weight(abstract_txt:language in 5973) [ClassicSimilarity], result of:
           0.047120497 = score(doc=5973,freq=1.0), product of:
             0.12018367 = queryWeight, product of:
               1.8083476 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015891703 = queryNorm
             0.3920707 = fieldWeight in 5973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.09375 = fieldNorm(doc=5973)
         0.1131988 = weight(abstract_txt:cross in 5973) [ClassicSimilarity], result of:
           0.1131988 = score(doc=5973,freq=1.0), product of:
             0.21557586 = queryWeight, product of:
               2.421917 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.015891703 = queryNorm
             0.52509964 = fieldWeight in 5973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.09375 = fieldNorm(doc=5973)
         0.41059244 = weight(abstract_txt:clef in 5973) [ClassicSimilarity], result of:
           0.41059244 = score(doc=5973,freq=1.0), product of:
             0.5089161 = queryWeight, product of:
               3.7211938 = boost
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.015891703 = queryNorm
             0.8067979 = fieldWeight in 5973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.6058445 = idf(docFreq=21, maxDocs=44218)
               0.09375 = fieldNorm(doc=5973)
         0.116321035 = weight(abstract_txt:evaluation in 5973) [ClassicSimilarity], result of:
           0.116321035 = score(doc=5973,freq=1.0), product of:
             0.27658018 = queryWeight, product of:
               3.87958 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015891703 = queryNorm
             0.42056894 = fieldWeight in 5973, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=5973)
       0.2 = coord(5/25)
   

5. Fujita, S.: NTCIR-2 as a Rosetta stone in laboratory experiments of IR systems (2005) 0.12

   0.12281599 = sum of:
     0.12281599 = product of:
       0.5117333 = sum of:
         0.058365945 = weight(abstract_txt:difficulties in 1017) [ClassicSimilarity], result of:
           0.058365945 = score(doc=1017,freq=1.0), product of:
             0.12423846 = queryWeight, product of:
               1.3000864 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.015891703 = queryNorm
             0.46978965 = fieldWeight in 1017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.078125 = fieldNorm(doc=1017)
         0.021322094 = weight(abstract_txt:systems in 1017) [ClassicSimilarity], result of:
           0.021322094 = score(doc=1017,freq=1.0), product of:
             0.07999195 = queryWeight, product of:
               1.4753067 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.015891703 = queryNorm
             0.26655298 = fieldWeight in 1017, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=1017)
         0.055532034 = weight(abstract_txt:language in 1017) [ClassicSimilarity], result of:
           0.055532034 = score(doc=1017,freq=2.0), product of:
             0.12018367 = queryWeight, product of:
               1.8083476 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.015891703 = queryNorm
             0.46205974 = fieldWeight in 1017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.078125 = fieldNorm(doc=1017)
         0.106021464 = weight(abstract_txt:languages in 1017) [ClassicSimilarity], result of:
           0.106021464 = score(doc=1017,freq=2.0), product of:
             0.18496048 = queryWeight, product of:
               2.2433584 = boost
               5.188118 = idf(docFreq=670, maxDocs=44218)
               0.015891703 = queryNorm
             0.57321143 = fieldWeight in 1017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.188118 = idf(docFreq=670, maxDocs=44218)
               0.078125 = fieldNorm(doc=1017)
         0.13340606 = weight(abstract_txt:cross in 1017) [ClassicSimilarity], result of:
           0.13340606 = score(doc=1017,freq=2.0), product of:
             0.21557586 = queryWeight, product of:
               2.421917 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.015891703 = queryNorm
             0.6188358 = fieldWeight in 1017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.078125 = fieldNorm(doc=1017)
         0.13708566 = weight(abstract_txt:evaluation in 1017) [ClassicSimilarity], result of:
           0.13708566 = score(doc=1017,freq=2.0), product of:
             0.27658018 = queryWeight, product of:
               3.87958 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.015891703 = queryNorm
             0.49564525 = fieldWeight in 1017, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=1017)
       0.24 = coord(6/25)