Document (#34021)

Author

Borlund, P.
Ruthven, I.

Title

Introduction to the special issue on evaluating interactive information retrieval systems

Source

Information processing and management. 44(2008) no.1, S.1-3

Year

2008

Abstract

Evaluation has always been a strong element of Information Retrieval (IR) research, much of our focus being on how we evaluate IR algorithms. As a research field we have benefited greatly from initiatives such as Cranfield, TREC, CLEF and INEX that have added to our knowledge of how to create test collections, the reliability of system-based evaluation criteria and our understanding of how to interpret the results of an algorithmic evaluation. In contrast, evaluations whose main focus is the user experience of searching have not yet reached the same level of maturity. Such evaluations are complex to create and assess due to the increased number of variables to incorporate within the study, the lack of standard tools available (for example, test collections) and the difficulty of selecting appropriate evaluation criteria for study. In spite of the complicated nature of user-centred evaluations, this form of evaluation is necessary to understand the effectiveness of individual IR systems and user search interactions. The growing incorporation of users into the evaluation process reflects the changing nature of IR within society; for example, more and more people have access to IR systems through Internet search engines but have little training or guidance in how to use these systems effectively. Similarly, new types of search system and new interactive IR facilities are becoming available to wide groups of end-users. In this special topic issue we present papers that tackle the methodological issues of evaluating interactive search systems. Methodologies can be presented at different levels; the papers by Blandford et al. and Petrelli present whole methodological approaches for evaluating interactive systems whereas those by Göker and Myrhaug and López Ostenero et al., consider what makes an appropriate evaluation methodological approach for specific retrieval situations. Any methodology must consider the nature of the methodological components, the instruments and processes by which we evaluate our systems. A number of papers have examined these issues in detail: Käki and Aula focus on specific methodological issues for the evaluation of Web search interfaces, Lopatovska and Mokros present alternate measures of retrieval success, Tenopir et al. examine the affective and cognitive verbalisations that occur within user studies and Kelly et al. analyse questionnaires, one of the basic tools for evaluations. The range of topics in this special issue as a whole nicely illustrates the variety and complexity by which user-centred evaluation of IR systems is undertaken.

Footnote

Einleitung eines Themenbereichs: Evaluation of Interactive Information Retrieval Systems

Similar documents (author)

1. Borlund, P.: ¬The IIR evaluation model : a framework for evaluation of interactive information retrieval systems (2003) 2.12

   2.1159768 = sum of:
     2.1159768 = product of:
       4.2319536 = sum of:
         4.2319536 = weight(author_txt:borlund in 1923) [ClassicSimilarity], result of:
           4.2319536 = score(doc=1923,freq=1.0), product of:
             0.7561414 = queryWeight, product of:
               1.0749224 = boost
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.07855395 = queryNorm
             5.5967755 = fieldWeight in 1923, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.625 = fieldNorm(doc=1923)
       0.5 = coord(1/2)
   

2. Borlund, P.: ¬The concept of relevance in IR (2003) 2.12

   2.1159768 = sum of:
     2.1159768 = product of:
       4.2319536 = sum of:
         4.2319536 = weight(author_txt:borlund in 2799) [ClassicSimilarity], result of:
           4.2319536 = score(doc=2799,freq=1.0), product of:
             0.7561414 = queryWeight, product of:
               1.0749224 = boost
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.07855395 = queryNorm
             5.5967755 = fieldWeight in 2799, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.625 = fieldNorm(doc=2799)
       0.5 = coord(1/2)
   

3. Borlund, P.: Evaluation of interactive information retrieval systems (2000) 2.12

   2.1159768 = sum of:
     2.1159768 = product of:
       4.2319536 = sum of:
         4.2319536 = weight(author_txt:borlund in 3557) [ClassicSimilarity], result of:
           4.2319536 = score(doc=3557,freq=1.0), product of:
             0.7561414 = queryWeight, product of:
               1.0749224 = boost
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.07855395 = queryNorm
             5.5967755 = fieldWeight in 3557, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.625 = fieldNorm(doc=3557)
       0.5 = coord(1/2)
   

4. Borlund, P.: Experimental components for the evaluation of interactive information retrieval systems (2000) 2.12

   2.1159768 = sum of:
     2.1159768 = product of:
       4.2319536 = sum of:
         4.2319536 = weight(author_txt:borlund in 550) [ClassicSimilarity], result of:
           4.2319536 = score(doc=550,freq=1.0), product of:
             0.7561414 = queryWeight, product of:
               1.0749224 = boost
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.07855395 = queryNorm
             5.5967755 = fieldWeight in 550, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.625 = fieldNorm(doc=550)
       0.5 = coord(1/2)
   

5. Borlund, P.: ¬A study of the use of simulated work task situations in interactive information retrieval evaluations : a meta-evaluation (2016) 2.12

   2.1159768 = sum of:
     2.1159768 = product of:
       4.2319536 = sum of:
         4.2319536 = weight(author_txt:borlund in 4881) [ClassicSimilarity], result of:
           4.2319536 = score(doc=4881,freq=1.0), product of:
             0.7561414 = queryWeight, product of:
               1.0749224 = boost
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.07855395 = queryNorm
             5.5967755 = fieldWeight in 4881, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.954841 = idf(docFreq=14, maxDocs=42740)
               0.625 = fieldNorm(doc=4881)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Tague-Sutcliffe, J.M.: Some perspectives on the evaluation of information retrieval systems (1996) 0.44

   0.4391346 = sum of:
     0.4391346 = product of:
       1.0978365 = sum of:
         0.059085127 = weight(abstract_txt:appropriate in 4232) [ClassicSimilarity], result of:
           0.059085127 = score(doc=4232,freq=1.0), product of:
             0.11825279 = queryWeight, product of:
               1.0352422 = boost
               5.329611 = idf(docFreq=562, maxDocs=42740)
               0.021432554 = queryNorm
             0.499651 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.329611 = idf(docFreq=562, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.082042016 = weight(abstract_txt:whole in 4232) [ClassicSimilarity], result of:
           0.082042016 = score(doc=4232,freq=1.0), product of:
             0.14718045 = queryWeight, product of:
               1.1549451 = boost
               5.945863 = idf(docFreq=303, maxDocs=42740)
               0.021432554 = queryNorm
             0.55742466 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.945863 = idf(docFreq=303, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.06493617 = weight(abstract_txt:retrieval in 4232) [ClassicSimilarity], result of:
           0.06493617 = score(doc=4232,freq=4.0), product of:
             0.09995559 = queryWeight, product of:
               1.3460302 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.021432554 = queryNorm
             0.6496502 = fieldWeight in 4232, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.07195964 = weight(abstract_txt:special in 4232) [ClassicSimilarity], result of:
           0.07195964 = score(doc=4232,freq=1.0), product of:
             0.15437691 = queryWeight, product of:
               1.4486821 = boost
               4.9720483 = idf(docFreq=804, maxDocs=42740)
               0.021432554 = queryNorm
             0.46612954 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9720483 = idf(docFreq=804, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.08078968 = weight(abstract_txt:issue in 4232) [ClassicSimilarity], result of:
           0.08078968 = score(doc=4232,freq=1.0), product of:
             0.16676067 = queryWeight, product of:
               1.5056663 = boost
               5.167625 = idf(docFreq=661, maxDocs=42740)
               0.021432554 = queryNorm
             0.48446482 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.167625 = idf(docFreq=661, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.0870669 = weight(abstract_txt:papers in 4232) [ClassicSimilarity], result of:
           0.0870669 = score(doc=4232,freq=1.0), product of:
             0.17529052 = queryWeight, product of:
               1.5436937 = boost
               5.2981396 = idf(docFreq=580, maxDocs=42740)
               0.021432554 = queryNorm
             0.49670058 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2981396 = idf(docFreq=580, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.23498365 = weight(abstract_txt:evaluating in 4232) [ClassicSimilarity], result of:
           0.23498365 = score(doc=4232,freq=4.0), product of:
             0.21405636 = queryWeight, product of:
               1.7058702 = boost
               5.8547482 = idf(docFreq=332, maxDocs=42740)
               0.021432554 = queryNorm
             1.0977653 = fieldWeight in 4232, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.8547482 = idf(docFreq=332, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.13338737 = weight(abstract_txt:interactive in 4232) [ClassicSimilarity], result of:
           0.13338737 = score(doc=4232,freq=1.0), product of:
             0.25639668 = queryWeight, product of:
               2.155795 = boost
               5.549208 = idf(docFreq=451, maxDocs=42740)
               0.021432554 = queryNorm
             0.5202383 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.549208 = idf(docFreq=451, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.12431111 = weight(abstract_txt:systems in 4232) [ClassicSimilarity], result of:
           0.12431111 = score(doc=4232,freq=4.0), product of:
             0.19416276 = queryWeight, product of:
               2.6530733 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.021432554 = queryNorm
             0.6402418 = fieldWeight in 4232, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
         0.15927482 = weight(abstract_txt:evaluation in 4232) [ClassicSimilarity], result of:
           0.15927482 = score(doc=4232,freq=1.0), product of:
             0.37814775 = queryWeight, product of:
               3.927112 = boost
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.021432554 = queryNorm
             0.4211973 = fieldWeight in 4232, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.09375 = fieldNorm(doc=4232)
       0.4 = coord(10/25)
   

2. Froehlich, T.J.: Relevance reconsidered : towards an agenda for the 21st century (1994) 0.35

   0.35043642 = sum of:
     0.35043642 = product of:
       0.79644644 = sum of:
         0.049237605 = weight(abstract_txt:appropriate in 6925) [ClassicSimilarity], result of:
           0.049237605 = score(doc=6925,freq=1.0), product of:
             0.11825279 = queryWeight, product of:
               1.0352422 = boost
               5.329611 = idf(docFreq=562, maxDocs=42740)
               0.021432554 = queryNorm
             0.41637585 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.329611 = idf(docFreq=562, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.03908977 = weight(abstract_txt:issues in 6925) [ClassicSimilarity], result of:
           0.03908977 = score(doc=6925,freq=1.0), product of:
             0.1160609 = queryWeight, product of:
               1.2561018 = boost
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.021432554 = queryNorm
             0.33680394 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.040920947 = weight(abstract_txt:present in 6925) [ClassicSimilarity], result of:
           0.040920947 = score(doc=6925,freq=1.0), product of:
             0.11965779 = queryWeight, product of:
               1.2754176 = boost
               4.377384 = idf(docFreq=1458, maxDocs=42740)
               0.021432554 = queryNorm
             0.34198314 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.377384 = idf(docFreq=1458, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.027056739 = weight(abstract_txt:retrieval in 6925) [ClassicSimilarity], result of:
           0.027056739 = score(doc=6925,freq=1.0), product of:
             0.09995559 = queryWeight, product of:
               1.3460302 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.021432554 = queryNorm
             0.2706876 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.055790614 = weight(abstract_txt:nature in 6925) [ClassicSimilarity], result of:
           0.055790614 = score(doc=6925,freq=1.0), product of:
             0.14712438 = queryWeight, product of:
               1.4142436 = boost
               4.8538513 = idf(docFreq=905, maxDocs=42740)
               0.021432554 = queryNorm
             0.37920713 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.8538513 = idf(docFreq=905, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.08480525 = weight(abstract_txt:special in 6925) [ClassicSimilarity], result of:
           0.08480525 = score(doc=6925,freq=2.0), product of:
             0.15437691 = queryWeight, product of:
               1.4486821 = boost
               4.9720483 = idf(docFreq=804, maxDocs=42740)
               0.021432554 = queryNorm
             0.54933894 = fieldWeight in 6925, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.9720483 = idf(docFreq=804, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.11660986 = weight(abstract_txt:issue in 6925) [ClassicSimilarity], result of:
           0.11660986 = score(doc=6925,freq=3.0), product of:
             0.16676067 = queryWeight, product of:
               1.5056663 = boost
               5.167625 = idf(docFreq=661, maxDocs=42740)
               0.021432554 = queryNorm
             0.69926476 = fieldWeight in 6925, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.167625 = idf(docFreq=661, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.07255576 = weight(abstract_txt:papers in 6925) [ClassicSimilarity], result of:
           0.07255576 = score(doc=6925,freq=1.0), product of:
             0.17529052 = queryWeight, product of:
               1.5436937 = boost
               5.2981396 = idf(docFreq=580, maxDocs=42740)
               0.021432554 = queryNorm
             0.41391715 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2981396 = idf(docFreq=580, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.032958906 = weight(abstract_txt:have in 6925) [ClassicSimilarity], result of:
           0.032958906 = score(doc=6925,freq=1.0), product of:
             0.13050757 = queryWeight, product of:
               1.8837131 = boost
               3.2325633 = idf(docFreq=4583, maxDocs=42740)
               0.021432554 = queryNorm
             0.25254402 = fieldWeight in 6925, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.2325633 = idf(docFreq=4583, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.08971382 = weight(abstract_txt:systems in 6925) [ClassicSimilarity], result of:
           0.08971382 = score(doc=6925,freq=3.0), product of:
             0.19416276 = queryWeight, product of:
               2.6530733 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.021432554 = queryNorm
             0.46205473 = fieldWeight in 6925, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
         0.18770716 = weight(abstract_txt:evaluation in 6925) [ClassicSimilarity], result of:
           0.18770716 = score(doc=6925,freq=2.0), product of:
             0.37814775 = queryWeight, product of:
               3.927112 = boost
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.021432554 = queryNorm
             0.49638575 = fieldWeight in 6925, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.078125 = fieldNorm(doc=6925)
       0.44 = coord(11/25)
   

3. Dunlop, M.D.; Johnson, C.W.; Reid, J.: Exploring the layers of information retrieval evaluation (1998) 0.34

   0.34403583 = sum of:
     0.34403583 = product of:
       0.9556551 = sum of:
         0.046907723 = weight(abstract_txt:issues in 4763) [ClassicSimilarity], result of:
           0.046907723 = score(doc=4763,freq=1.0), product of:
             0.1160609 = queryWeight, product of:
               1.2561018 = boost
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.021432554 = queryNorm
             0.40416473 = fieldWeight in 4763, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.04591681 = weight(abstract_txt:retrieval in 4763) [ClassicSimilarity], result of:
           0.04591681 = score(doc=4763,freq=2.0), product of:
             0.09995559 = queryWeight, product of:
               1.3460302 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.021432554 = queryNorm
             0.4593721 = fieldWeight in 4763, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.07195964 = weight(abstract_txt:special in 4763) [ClassicSimilarity], result of:
           0.07195964 = score(doc=4763,freq=1.0), product of:
             0.15437691 = queryWeight, product of:
               1.4486821 = boost
               4.9720483 = idf(docFreq=804, maxDocs=42740)
               0.021432554 = queryNorm
             0.46612954 = fieldWeight in 4763, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9720483 = idf(docFreq=804, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.08078968 = weight(abstract_txt:issue in 4763) [ClassicSimilarity], result of:
           0.08078968 = score(doc=4763,freq=1.0), product of:
             0.16676067 = queryWeight, product of:
               1.5056663 = boost
               5.167625 = idf(docFreq=661, maxDocs=42740)
               0.021432554 = queryNorm
             0.48446482 = fieldWeight in 4763, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.167625 = idf(docFreq=661, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.12313119 = weight(abstract_txt:papers in 4763) [ClassicSimilarity], result of:
           0.12313119 = score(doc=4763,freq=2.0), product of:
             0.17529052 = queryWeight, product of:
               1.5436937 = boost
               5.2981396 = idf(docFreq=580, maxDocs=42740)
               0.021432554 = queryNorm
             0.7024407 = fieldWeight in 4763, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.2981396 = idf(docFreq=580, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.16615851 = weight(abstract_txt:evaluating in 4763) [ClassicSimilarity], result of:
           0.16615851 = score(doc=4763,freq=2.0), product of:
             0.21405636 = queryWeight, product of:
               1.7058702 = boost
               5.8547482 = idf(docFreq=332, maxDocs=42740)
               0.021432554 = queryNorm
             0.77623725 = fieldWeight in 4763, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8547482 = idf(docFreq=332, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.13338737 = weight(abstract_txt:interactive in 4763) [ClassicSimilarity], result of:
           0.13338737 = score(doc=4763,freq=1.0), product of:
             0.25639668 = queryWeight, product of:
               2.155795 = boost
               5.549208 = idf(docFreq=451, maxDocs=42740)
               0.021432554 = queryNorm
             0.5202383 = fieldWeight in 4763, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.549208 = idf(docFreq=451, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.062155556 = weight(abstract_txt:systems in 4763) [ClassicSimilarity], result of:
           0.062155556 = score(doc=4763,freq=1.0), product of:
             0.19416276 = queryWeight, product of:
               2.6530733 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.021432554 = queryNorm
             0.3201209 = fieldWeight in 4763, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
         0.2252486 = weight(abstract_txt:evaluation in 4763) [ClassicSimilarity], result of:
           0.2252486 = score(doc=4763,freq=2.0), product of:
             0.37814775 = queryWeight, product of:
               3.927112 = boost
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.021432554 = queryNorm
             0.59566295 = fieldWeight in 4763, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.09375 = fieldNorm(doc=4763)
       0.36 = coord(9/25)
   

4. Díaz, A.; García, A.; Gervás, P.: User-centred versus system-centred evaluation of a personalization system (2008) 0.34

   0.3407437 = sum of:
     0.3407437 = product of:
       0.94651026 = sum of:
         0.04032216 = weight(abstract_txt:evaluate in 4095) [ClassicSimilarity], result of:
           0.04032216 = score(doc=4095,freq=1.0), product of:
             0.12011096 = queryWeight, product of:
               1.0433441 = boost
               5.371321 = idf(docFreq=539, maxDocs=42740)
               0.021432554 = queryNorm
             0.33570758 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.371321 = idf(docFreq=539, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.054694675 = weight(abstract_txt:whole in 4095) [ClassicSimilarity], result of:
           0.054694675 = score(doc=4095,freq=1.0), product of:
             0.14718045 = queryWeight, product of:
               1.1549451 = boost
               5.945863 = idf(docFreq=303, maxDocs=42740)
               0.021432554 = queryNorm
             0.37161642 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.945863 = idf(docFreq=303, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.031271815 = weight(abstract_txt:issues in 4095) [ClassicSimilarity], result of:
           0.031271815 = score(doc=4095,freq=1.0), product of:
             0.1160609 = queryWeight, product of:
               1.2561018 = boost
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.021432554 = queryNorm
             0.26944315 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.3110905 = idf(docFreq=1558, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.29272556 = weight(abstract_txt:centred in 4095) [ClassicSimilarity], result of:
           0.29272556 = score(doc=4095,freq=7.0), product of:
             0.23541163 = queryWeight, product of:
               1.4606637 = boost
               7.519756 = idf(docFreq=62, maxDocs=42740)
               0.021432554 = queryNorm
             1.2434627 = fieldWeight in 4095, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               7.519756 = idf(docFreq=62, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.079393655 = weight(abstract_txt:user in 4095) [ClassicSimilarity], result of:
           0.079393655 = score(doc=4095,freq=6.0), product of:
             0.14093111 = queryWeight, product of:
               1.7869389 = boost
               3.6797917 = idf(docFreq=2930, maxDocs=42740)
               0.021432554 = queryNorm
             0.5633508 = fieldWeight in 4095, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.6797917 = idf(docFreq=2930, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.037288744 = weight(abstract_txt:have in 4095) [ClassicSimilarity], result of:
           0.037288744 = score(doc=4095,freq=2.0), product of:
             0.13050757 = queryWeight, product of:
               1.8837131 = boost
               3.2325633 = idf(docFreq=4583, maxDocs=42740)
               0.021432554 = queryNorm
             0.2857209 = fieldWeight in 4095, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.2325633 = idf(docFreq=4583, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.15701017 = weight(abstract_txt:evaluations in 4095) [ClassicSimilarity], result of:
           0.15701017 = score(doc=4095,freq=1.0), product of:
             0.37455475 = queryWeight, product of:
               2.605607 = boost
               6.7070637 = idf(docFreq=141, maxDocs=42740)
               0.021432554 = queryNorm
             0.41919148 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.7070637 = idf(docFreq=141, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.041437037 = weight(abstract_txt:systems in 4095) [ClassicSimilarity], result of:
           0.041437037 = score(doc=4095,freq=1.0), product of:
             0.19416276 = queryWeight, product of:
               2.6530733 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.021432554 = queryNorm
             0.21341394 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
         0.21236642 = weight(abstract_txt:evaluation in 4095) [ClassicSimilarity], result of:
           0.21236642 = score(doc=4095,freq=4.0), product of:
             0.37814775 = queryWeight, product of:
               3.927112 = boost
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.021432554 = queryNorm
             0.5615964 = fieldWeight in 4095, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.0625 = fieldNorm(doc=4095)
       0.36 = coord(9/25)
   

5. Blandford, A.; Adams, A.; Attfield, S.; Buchanan, G.; Gow, J.; Makri, S.; Rimmer, J.; Warwick, C.: ¬The PRET A Rapporter framework : evaluating digital libraries from the perspective of information work (2008) 0.32

   0.3169998 = sum of:
     0.3169998 = product of:
       0.99062437 = sum of:
         0.056668006 = weight(abstract_txt:criteria in 4022) [ClassicSimilarity], result of:
           0.056668006 = score(doc=4022,freq=1.0), product of:
             0.12986897 = queryWeight, product of:
               1.0848981 = boost
               5.5852485 = idf(docFreq=435, maxDocs=42740)
               0.021432554 = queryNorm
             0.43634754 = fieldWeight in 4022, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5852485 = idf(docFreq=435, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.040920947 = weight(abstract_txt:present in 4022) [ClassicSimilarity], result of:
           0.040920947 = score(doc=4022,freq=1.0), product of:
             0.11965779 = queryWeight, product of:
               1.2754176 = boost
               4.377384 = idf(docFreq=1458, maxDocs=42740)
               0.021432554 = queryNorm
             0.34198314 = fieldWeight in 4022, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.377384 = idf(docFreq=1458, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.08217027 = weight(abstract_txt:focus in 4022) [ClassicSimilarity], result of:
           0.08217027 = score(doc=4022,freq=2.0), product of:
             0.15116237 = queryWeight, product of:
               1.43352 = boost
               4.92001 = idf(docFreq=847, maxDocs=42740)
               0.021432554 = queryNorm
             0.5435895 = fieldWeight in 4022, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.92001 = idf(docFreq=847, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.13829984 = weight(abstract_txt:centred in 4022) [ClassicSimilarity], result of:
           0.13829984 = score(doc=4022,freq=1.0), product of:
             0.23541163 = queryWeight, product of:
               1.4606637 = boost
               7.519756 = idf(docFreq=62, maxDocs=42740)
               0.021432554 = queryNorm
             0.5874809 = fieldWeight in 4022, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.519756 = idf(docFreq=62, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.07017474 = weight(abstract_txt:user in 4022) [ClassicSimilarity], result of:
           0.07017474 = score(doc=4022,freq=3.0), product of:
             0.14093111 = queryWeight, product of:
               1.7869389 = boost
               3.6797917 = idf(docFreq=2930, maxDocs=42740)
               0.021432554 = queryNorm
             0.49793643 = fieldWeight in 4022, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6797917 = idf(docFreq=2930, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.111156136 = weight(abstract_txt:interactive in 4022) [ClassicSimilarity], result of:
           0.111156136 = score(doc=4022,freq=1.0), product of:
             0.25639668 = queryWeight, product of:
               2.155795 = boost
               5.549208 = idf(docFreq=451, maxDocs=42740)
               0.021432554 = queryNorm
             0.43353188 = fieldWeight in 4022, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.549208 = idf(docFreq=451, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.11582005 = weight(abstract_txt:systems in 4022) [ClassicSimilarity], result of:
           0.11582005 = score(doc=4022,freq=5.0), product of:
             0.19416276 = queryWeight, product of:
               2.6530733 = boost
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.021432554 = queryNorm
             0.5965101 = fieldWeight in 4022, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.414623 = idf(docFreq=3820, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
         0.3754143 = weight(abstract_txt:evaluation in 4022) [ClassicSimilarity], result of:
           0.3754143 = score(doc=4022,freq=8.0), product of:
             0.37814775 = queryWeight, product of:
               3.927112 = boost
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.021432554 = queryNorm
             0.9927715 = fieldWeight in 4022, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.492771 = idf(docFreq=1299, maxDocs=42740)
               0.078125 = fieldNorm(doc=4022)
       0.32 = coord(8/25)