Document (#34028)

Author
Petrelli, D.
Title
On the role of user-centred evaluation in the advancement of interactive information retrieval
Source
Information processing and management. 44(2008) no.1, S.22-38
Year
2008
Abstract
This paper discusses the role of user-centred evaluations as an essential method for researching interactive information retrieval. It draws mainly on the work carried out during the Clarity Project where different user-centred evaluations were run during the lifecycle of a cross-language information retrieval system. The iterative testing was not only instrumental to the development of a usable system, but it enhanced our knowledge of the potential, impact, and actual use of cross-language information retrieval technology. Indeed the role of the user evaluation was dual: by testing a specific prototype it was possible to gain a micro-view and assess the effectiveness of each component of the complex system; by cumulating the result of all the evaluations (in total 43 people were involved) it was possible to build a macro-view of how cross-language retrieval would impact on users and their tasks. By showing the richness of results that can be acquired, this paper aims at stimulating researchers into considering user-centred evaluations as a flexible, adaptable and comprehensive technique for investigating non-traditional information access systems.
Footnote
Beitrag eines Themenbereichs: Evaluation of Interactive Information Retrieval Systems
Theme
Retrievalstudien

Similar documents (content)

  1. Petrelli, D.; Levin, S.; Beaulieu, M.; Sanderson, M.: Which user interaction for cross-language information retrieval? : design issues and reflections (2006) 0.35
    0.34976625 = sum of:
      0.34976625 = product of:
        0.87441564 = sum of:
          0.099946454 = weight(abstract_txt:instrumental in 54) [ClassicSimilarity], result of:
            0.099946454 = score(doc=54,freq=1.0), product of:
              0.1581094 = queryWeight, product of:
                1.0312408 = boost
                8.091326 = idf(docFreq=35, maxDocs=43254)
                0.018948633 = queryNorm
              0.6321348 = fieldWeight in 54, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                8.091326 = idf(docFreq=35, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.04376945 = weight(abstract_txt:during in 54) [ClassicSimilarity], result of:
            0.04376945 = score(doc=54,freq=1.0), product of:
              0.114877984 = queryWeight, product of:
                1.2431246 = boost
                4.8769045 = idf(docFreq=895, maxDocs=43254)
                0.018948633 = queryNorm
              0.38100815 = fieldWeight in 54, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8769045 = idf(docFreq=895, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.04768272 = weight(abstract_txt:view in 54) [ClassicSimilarity], result of:
            0.04768272 = score(doc=54,freq=1.0), product of:
              0.12162703 = queryWeight, product of:
                1.2791201 = boost
                5.0181184 = idf(docFreq=777, maxDocs=43254)
                0.018948633 = queryNorm
              0.3920405 = fieldWeight in 54, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.0181184 = idf(docFreq=777, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.03049119 = weight(abstract_txt:system in 54) [ClassicSimilarity], result of:
            0.03049119 = score(doc=54,freq=2.0), product of:
              0.0820212 = queryWeight, product of:
                1.2864857 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018948633 = queryNorm
              0.37174767 = fieldWeight in 54, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.019070394 = weight(abstract_txt:information in 54) [ClassicSimilarity], result of:
            0.019070394 = score(doc=54,freq=2.0), product of:
              0.07112122 = queryWeight, product of:
                1.5465568 = boost
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.018948633 = queryNorm
              0.2681393 = fieldWeight in 54, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.08340154 = weight(abstract_txt:language in 54) [ClassicSimilarity], result of:
            0.08340154 = score(doc=54,freq=4.0), product of:
              0.12732564 = queryWeight, product of:
                1.6028756 = boost
                4.192163 = idf(docFreq=1776, maxDocs=43254)
                0.018948633 = queryNorm
              0.6550255 = fieldWeight in 54, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.192163 = idf(docFreq=1776, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.17362286 = weight(abstract_txt:cross in 54) [ClassicSimilarity], result of:
            0.17362286 = score(doc=54,freq=3.0), product of:
              0.22848131 = queryWeight, product of:
                2.147174 = boost
                5.6157217 = idf(docFreq=427, maxDocs=43254)
                0.018948633 = queryNorm
              0.7598996 = fieldWeight in 54, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.6157217 = idf(docFreq=427, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.068263695 = weight(abstract_txt:retrieval in 54) [ClassicSimilarity], result of:
            0.068263695 = score(doc=54,freq=3.0), product of:
              0.14538588 = queryWeight, product of:
                2.2111967 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.018948633 = queryNorm
              0.46953458 = fieldWeight in 54, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.08137661 = weight(abstract_txt:user in 54) [ClassicSimilarity], result of:
            0.08137661 = score(doc=54,freq=3.0), product of:
              0.16345395 = queryWeight, product of:
                2.3445742 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018948633 = queryNorm
              0.4978565 = fieldWeight in 54, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
          0.22679076 = weight(abstract_txt:evaluations in 54) [ClassicSimilarity], result of:
            0.22679076 = score(doc=54,freq=1.0), product of:
              0.43339407 = queryWeight, product of:
                3.4147015 = boost
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.018948633 = queryNorm
              0.5232899 = fieldWeight in 54, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.078125 = fieldNorm(doc=54)
        0.4 = coord(10/25)
    
  2. Díaz, A.; García, A.; Gervás, P.: User-centred versus system-centred evaluation of a personalization system (2008) 0.31
    0.30858076 = sum of:
      0.30858076 = product of:
        1.1020741 = sum of:
          0.054658778 = weight(abstract_txt:evaluation in 4095) [ClassicSimilarity], result of:
            0.054658778 = score(doc=4095,freq=4.0), product of:
              0.097382896 = queryWeight, product of:
                1.1445575 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018948633 = queryNorm
              0.561277 = fieldWeight in 4095, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
          0.03501556 = weight(abstract_txt:during in 4095) [ClassicSimilarity], result of:
            0.03501556 = score(doc=4095,freq=1.0), product of:
              0.114877984 = queryWeight, product of:
                1.2431246 = boost
                4.8769045 = idf(docFreq=895, maxDocs=43254)
                0.018948633 = queryNorm
              0.30480653 = fieldWeight in 4095, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.8769045 = idf(docFreq=895, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
          0.045635037 = weight(abstract_txt:system in 4095) [ClassicSimilarity], result of:
            0.045635037 = score(doc=4095,freq=7.0), product of:
              0.0820212 = queryWeight, product of:
                1.2864857 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018948633 = queryNorm
              0.556381 = fieldWeight in 4095, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
          0.010787845 = weight(abstract_txt:information in 4095) [ClassicSimilarity], result of:
            0.010787845 = score(doc=4095,freq=1.0), product of:
              0.07112122 = queryWeight, product of:
                1.5465568 = boost
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.018948633 = queryNorm
              0.1516825 = fieldWeight in 4095, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
          0.09206714 = weight(abstract_txt:user in 4095) [ClassicSimilarity], result of:
            0.09206714 = score(doc=4095,freq=6.0), product of:
              0.16345395 = queryWeight, product of:
                2.3445742 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018948633 = queryNorm
              0.5632604 = fieldWeight in 4095, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
          0.1814326 = weight(abstract_txt:evaluations in 4095) [ClassicSimilarity], result of:
            0.1814326 = score(doc=4095,freq=1.0), product of:
              0.43339407 = queryWeight, product of:
                3.4147015 = boost
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.018948633 = queryNorm
              0.41863194 = fieldWeight in 4095, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
          0.6824771 = weight(abstract_txt:centred in 4095) [ClassicSimilarity], result of:
            0.6824771 = score(doc=4095,freq=7.0), product of:
              0.54798096 = queryWeight, product of:
                3.839671 = boost
                7.53171 = idf(docFreq=62, maxDocs=43254)
                0.018948633 = queryNorm
              1.2454394 = fieldWeight in 4095, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                7.53171 = idf(docFreq=62, maxDocs=43254)
                0.0625 = fieldNorm(doc=4095)
        0.28 = coord(7/25)
    
  3. Borlund, P.; Ruthven, I.: Introduction to the special issue on evaluating interactive information retrieval systems (2008) 0.26
    0.25751743 = sum of:
      0.25751743 = product of:
        0.804742 = sum of:
          0.061491124 = weight(abstract_txt:evaluation in 4020) [ClassicSimilarity], result of:
            0.061491124 = score(doc=4020,freq=9.0), product of:
              0.097382896 = queryWeight, product of:
                1.1445575 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018948633 = queryNorm
              0.6314366 = fieldWeight in 4020, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.018294714 = weight(abstract_txt:system in 4020) [ClassicSimilarity], result of:
            0.018294714 = score(doc=4020,freq=2.0), product of:
              0.0820212 = queryWeight, product of:
                1.2864857 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018948633 = queryNorm
              0.22304861 = fieldWeight in 4020, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.06712452 = weight(abstract_txt:interactive in 4020) [ClassicSimilarity], result of:
            0.06712452 = score(doc=4020,freq=3.0), product of:
              0.14890261 = queryWeight, product of:
                1.4152964 = boost
                5.552352 = idf(docFreq=455, maxDocs=43254)
                0.018948633 = queryNorm
              0.45079482 = fieldWeight in 4020, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.552352 = idf(docFreq=455, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.0080908835 = weight(abstract_txt:information in 4020) [ClassicSimilarity], result of:
            0.0080908835 = score(doc=4020,freq=1.0), product of:
              0.07112122 = queryWeight, product of:
                1.5465568 = boost
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.018948633 = queryNorm
              0.11376187 = fieldWeight in 4020, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.04095822 = weight(abstract_txt:retrieval in 4020) [ClassicSimilarity], result of:
            0.04095822 = score(doc=4020,freq=3.0), product of:
              0.14538588 = queryWeight, product of:
                2.2111967 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.018948633 = queryNorm
              0.28172076 = fieldWeight in 4020, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.063034065 = weight(abstract_txt:user in 4020) [ClassicSimilarity], result of:
            0.063034065 = score(doc=4020,freq=5.0), product of:
              0.16345395 = queryWeight, product of:
                2.3445742 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018948633 = queryNorm
              0.38563806 = fieldWeight in 4020, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.2721489 = weight(abstract_txt:evaluations in 4020) [ClassicSimilarity], result of:
            0.2721489 = score(doc=4020,freq=4.0), product of:
              0.43339407 = queryWeight, product of:
                3.4147015 = boost
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.018948633 = queryNorm
              0.6279479 = fieldWeight in 4020, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
          0.2735995 = weight(abstract_txt:centred in 4020) [ClassicSimilarity], result of:
            0.2735995 = score(doc=4020,freq=2.0), product of:
              0.54798096 = queryWeight, product of:
                3.839671 = boost
                7.53171 = idf(docFreq=62, maxDocs=43254)
                0.018948633 = queryNorm
              0.49928653 = fieldWeight in 4020, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.53171 = idf(docFreq=62, maxDocs=43254)
                0.046875 = fieldNorm(doc=4020)
        0.32 = coord(8/25)
    
  4. TREC: experiment and evaluation in information retrieval (2005) 0.18
    0.17701824 = sum of:
      0.17701824 = product of:
        0.553182 = sum of:
          0.038649593 = weight(abstract_txt:evaluation in 1762) [ClassicSimilarity], result of:
            0.038649593 = score(doc=1762,freq=2.0), product of:
              0.097382896 = queryWeight, product of:
                1.1445575 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018948633 = queryNorm
              0.39688274 = fieldWeight in 1762, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.029577086 = weight(abstract_txt:impact in 1762) [ClassicSimilarity], result of:
            0.029577086 = score(doc=1762,freq=1.0), product of:
              0.102651745 = queryWeight, product of:
                1.1751125 = boost
                4.610086 = idf(docFreq=1169, maxDocs=43254)
                0.018948633 = queryNorm
              0.28813037 = fieldWeight in 1762, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.610086 = idf(docFreq=1169, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.05167248 = weight(abstract_txt:interactive in 1762) [ClassicSimilarity], result of:
            0.05167248 = score(doc=1762,freq=1.0), product of:
              0.14890261 = queryWeight, product of:
                1.4152964 = boost
                5.552352 = idf(docFreq=455, maxDocs=43254)
                0.018948633 = queryNorm
              0.347022 = fieldWeight in 1762, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.552352 = idf(docFreq=455, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.015256316 = weight(abstract_txt:information in 1762) [ClassicSimilarity], result of:
            0.015256316 = score(doc=1762,freq=2.0), product of:
              0.07112122 = queryWeight, product of:
                1.5465568 = boost
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.018948633 = queryNorm
              0.21451144 = fieldWeight in 1762, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.047179032 = weight(abstract_txt:language in 1762) [ClassicSimilarity], result of:
            0.047179032 = score(doc=1762,freq=2.0), product of:
              0.12732564 = queryWeight, product of:
                1.6028756 = boost
                4.192163 = idf(docFreq=1776, maxDocs=43254)
                0.018948633 = queryNorm
              0.37053835 = fieldWeight in 1762, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.192163 = idf(docFreq=1776, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.08019297 = weight(abstract_txt:cross in 1762) [ClassicSimilarity], result of:
            0.08019297 = score(doc=1762,freq=1.0), product of:
              0.22848131 = queryWeight, product of:
                2.147174 = boost
                5.6157217 = idf(docFreq=427, maxDocs=43254)
                0.018948633 = queryNorm
              0.3509826 = fieldWeight in 1762, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.6157217 = idf(docFreq=427, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.10922191 = weight(abstract_txt:retrieval in 1762) [ClassicSimilarity], result of:
            0.10922191 = score(doc=1762,freq=12.0), product of:
              0.14538588 = queryWeight, product of:
                2.2111967 = boost
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.018948633 = queryNorm
              0.75125533 = fieldWeight in 1762, product of:
                3.4641016 = tf(freq=12.0), with freq of:
                  12.0 = termFreq=12.0
                3.4699 = idf(docFreq=3658, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
          0.1814326 = weight(abstract_txt:evaluations in 1762) [ClassicSimilarity], result of:
            0.1814326 = score(doc=1762,freq=1.0), product of:
              0.43339407 = queryWeight, product of:
                3.4147015 = boost
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.018948633 = queryNorm
              0.41863194 = fieldWeight in 1762, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.0625 = fieldNorm(doc=1762)
        0.32 = coord(8/25)
    
  5. Gross, M.: ¬The imposed query : implications for library service evaluation (1998) 0.17
    0.16991554 = sum of:
      0.16991554 = product of:
        0.5309861 = sum of:
          0.06832347 = weight(abstract_txt:evaluation in 5703) [ClassicSimilarity], result of:
            0.06832347 = score(doc=5703,freq=4.0), product of:
              0.097382896 = queryWeight, product of:
                1.1445575 = boost
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.018948633 = queryNorm
              0.7015962 = fieldWeight in 5703, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.490216 = idf(docFreq=1318, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.036971357 = weight(abstract_txt:impact in 5703) [ClassicSimilarity], result of:
            0.036971357 = score(doc=5703,freq=1.0), product of:
              0.102651745 = queryWeight, product of:
                1.1751125 = boost
                4.610086 = idf(docFreq=1169, maxDocs=43254)
                0.018948633 = queryNorm
              0.36016297 = fieldWeight in 5703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.610086 = idf(docFreq=1169, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.037535924 = weight(abstract_txt:possible in 5703) [ClassicSimilarity], result of:
            0.037535924 = score(doc=5703,freq=1.0), product of:
              0.103694126 = queryWeight, product of:
                1.1810638 = boost
                4.6334333 = idf(docFreq=1142, maxDocs=43254)
                0.018948633 = queryNorm
              0.361987 = fieldWeight in 5703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6334333 = idf(docFreq=1142, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.021560527 = weight(abstract_txt:system in 5703) [ClassicSimilarity], result of:
            0.021560527 = score(doc=5703,freq=1.0), product of:
              0.0820212 = queryWeight, product of:
                1.2864857 = boost
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.018948633 = queryNorm
              0.2628653 = fieldWeight in 5703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.364676 = idf(docFreq=4064, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.023356367 = weight(abstract_txt:information in 5703) [ClassicSimilarity], result of:
            0.023356367 = score(doc=5703,freq=3.0), product of:
              0.07112122 = queryWeight, product of:
                1.5465568 = boost
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.018948633 = queryNorm
              0.32840222 = fieldWeight in 5703, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                2.42692 = idf(docFreq=10382, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.050003935 = weight(abstract_txt:role in 5703) [ClassicSimilarity], result of:
            0.050003935 = score(doc=5703,freq=1.0), product of:
              0.14371076 = queryWeight, product of:
                1.7028898 = boost
                4.45374 = idf(docFreq=1367, maxDocs=43254)
                0.018948633 = queryNorm
              0.34794843 = fieldWeight in 5703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.45374 = idf(docFreq=1367, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.066443734 = weight(abstract_txt:user in 5703) [ClassicSimilarity], result of:
            0.066443734 = score(doc=5703,freq=2.0), product of:
              0.16345395 = queryWeight, product of:
                2.3445742 = boost
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.018948633 = queryNorm
              0.40649816 = fieldWeight in 5703, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.6792014 = idf(docFreq=2967, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
          0.22679076 = weight(abstract_txt:evaluations in 5703) [ClassicSimilarity], result of:
            0.22679076 = score(doc=5703,freq=1.0), product of:
              0.43339407 = queryWeight, product of:
                3.4147015 = boost
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.018948633 = queryNorm
              0.5232899 = fieldWeight in 5703, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.698111 = idf(docFreq=144, maxDocs=43254)
                0.078125 = fieldNorm(doc=5703)
        0.32 = coord(8/25)