Document (#36745)

Author

Sun, Y.
Kantor, P.B.
Morse, E.L.

Title

Using cross-evaluation to evaluate interactive QA systems

Source

Journal of the American Society for Information Science and Technology. 62(2011) no.9, S.1653-1665

Year

2011

Abstract

In this article, we report on an experiment to assess the possibility of rigorous evaluation of interactive question-answering (QA) systems using the cross-evaluation method. This method takes into account the effects of tasks and context, and of the users of the systems. Statistical techniques are used to remove these effects, isolating the effect of the system itself. The results show that this approach yields meaningful measurements of the impact of systems on user task performance, using a surprisingly small number of subjects and without relying on predetermined judgments of the quality, or of the relevance of materials. We conclude that the method is indeed effective for comparing end-to-end QA systems, and for comparing interactive systems with high efficiency.

Similar documents (author)

1. Morse, P.M.: Search theory and browsing (1970) 2.44

   2.4370155 = sum of:
     2.4370155 = product of:
       4.874031 = sum of:
         4.874031 = weight(author_txt:morse in 1448) [ClassicSimilarity], result of:
           4.874031 = score(doc=1448,freq=1.0), product of:
             0.8108649 = queryWeight, product of:
               1.1770902 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.07162737 = queryNorm
             6.010904 = fieldWeight in 1448, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.625 = fieldNorm(doc=1448)
       0.5 = coord(1/2)
   

2. Morse, P.M.: Browsing and search theory (1973) 2.44

   2.4370155 = sum of:
     2.4370155 = product of:
       4.874031 = sum of:
         4.874031 = weight(author_txt:morse in 3339) [ClassicSimilarity], result of:
           4.874031 = score(doc=3339,freq=1.0), product of:
             0.8108649 = queryWeight, product of:
               1.1770902 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.07162737 = queryNorm
             6.010904 = fieldWeight in 3339, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.625 = fieldNorm(doc=3339)
       0.5 = coord(1/2)
   

3. Morse, T.: Mapping relationships : examining bibliographic relationships in sheet maps from Tillett to RDA (2012) 2.44

   2.4370155 = sum of:
     2.4370155 = product of:
       4.874031 = sum of:
         4.874031 = weight(author_txt:morse in 1896) [ClassicSimilarity], result of:
           4.874031 = score(doc=1896,freq=1.0), product of:
             0.8108649 = queryWeight, product of:
               1.1770902 = boost
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.07162737 = queryNorm
             6.010904 = fieldWeight in 1896, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.617446 = idf(docFreq=7, maxDocs=44218)
               0.625 = fieldNorm(doc=1896)
       0.5 = coord(1/2)
   

4. Kantor, P.B.: ¬The Adaptive Network Library Interface : a historical overview and interim report (1993) 1.49

   1.4942701 = sum of:
     1.4942701 = product of:
       2.9885402 = sum of:
         2.9885402 = weight(author_txt:kantor in 6976) [ClassicSimilarity], result of:
           2.9885402 = score(doc=6976,freq=1.0), product of:
             0.58523333 = queryWeight, product of:
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.07162737 = queryNorm
             5.106579 = fieldWeight in 6976, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.625 = fieldNorm(doc=6976)
       0.5 = coord(1/2)
   

5. Kantor, P.B.: ¬The logic of weighted queries (1981) 1.49

   1.4942701 = sum of:
     1.4942701 = product of:
       2.9885402 = sum of:
         2.9885402 = weight(author_txt:kantor in 7622) [ClassicSimilarity], result of:
           2.9885402 = score(doc=7622,freq=1.0), product of:
             0.58523333 = queryWeight, product of:
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.07162737 = queryNorm
             5.106579 = fieldWeight in 7622, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.625 = fieldNorm(doc=7622)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Wacholder, N.; Kelly, D.; Kantor, P.; Rittman, R.; Sun, Y.; Bai, B.; Small, S.; Yamrom, B.; Strzalkowski, T.: ¬A model for quantitative evaluation of an end-to-end question-answering system (2007) 0.69

   0.6858859 = sum of:
     0.6858859 = product of:
       1.428929 = sum of:
         0.06790622 = weight(abstract_txt:conclude in 435) [ClassicSimilarity], result of:
           0.06790622 = score(doc=435,freq=1.0), product of:
             0.13709122 = queryWeight, product of:
               1.0477343 = boost
               6.340301 = idf(docFreq=211, maxDocs=44218)
               0.020637099 = queryNorm
             0.49533603 = fieldWeight in 435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.340301 = idf(docFreq=211, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.11195915 = weight(abstract_txt:answering in 435) [ClassicSimilarity], result of:
           0.11195915 = score(doc=435,freq=2.0), product of:
             0.15185636 = queryWeight, product of:
               1.1027138 = boost
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.020637099 = queryNorm
             0.73727006 = fieldWeight in 435, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.087252825 = weight(abstract_txt:judgments in 435) [ClassicSimilarity], result of:
           0.087252825 = score(doc=435,freq=1.0), product of:
             0.16202782 = queryWeight, product of:
               1.1390456 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.020637099 = queryNorm
             0.5385052 = fieldWeight in 435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.022460148 = weight(abstract_txt:this in 435) [ClassicSimilarity], result of:
           0.022460148 = score(doc=435,freq=4.0), product of:
             0.059570618 = queryWeight, product of:
               1.1962528 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.020637099 = queryNorm
             0.37703398 = fieldWeight in 435, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.11483338 = weight(abstract_txt:surprisingly in 435) [ClassicSimilarity], result of:
           0.11483338 = score(doc=435,freq=1.0), product of:
             0.1945877 = queryWeight, product of:
               1.2482569 = boost
               7.5537524 = idf(docFreq=62, maxDocs=44218)
               0.020637099 = queryNorm
             0.5901369 = fieldWeight in 435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5537524 = idf(docFreq=62, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.18196684 = weight(abstract_txt:predetermined in 435) [ClassicSimilarity], result of:
           0.18196684 = score(doc=435,freq=1.0), product of:
             0.2644832 = queryWeight, product of:
               1.455276 = boost
               8.806516 = idf(docFreq=17, maxDocs=44218)
               0.020637099 = queryNorm
             0.688009 = fieldWeight in 435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.806516 = idf(docFreq=17, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.03319734 = weight(abstract_txt:using in 435) [ClassicSimilarity], result of:
           0.03319734 = score(doc=435,freq=1.0), product of:
             0.12270037 = queryWeight, product of:
               1.7168401 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.020637099 = queryNorm
             0.27055615 = fieldWeight in 435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.11829003 = weight(abstract_txt:comparing in 435) [ClassicSimilarity], result of:
           0.11829003 = score(doc=435,freq=1.0), product of:
             0.25006065 = queryWeight, product of:
               2.00117 = boost
               6.0549803 = idf(docFreq=281, maxDocs=44218)
               0.020637099 = queryNorm
             0.47304535 = fieldWeight in 435, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0549803 = idf(docFreq=281, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.14432082 = weight(abstract_txt:evaluation in 435) [ClassicSimilarity], result of:
           0.14432082 = score(doc=435,freq=4.0), product of:
             0.20589368 = queryWeight, product of:
               2.2239676 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.020637099 = queryNorm
             0.70094824 = fieldWeight in 435, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.14576179 = weight(abstract_txt:method in 435) [ClassicSimilarity], result of:
           0.14576179 = score(doc=435,freq=4.0), product of:
             0.2072619 = queryWeight, product of:
               2.2313447 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.020637099 = queryNorm
             0.7032734 = fieldWeight in 435, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.27399844 = weight(abstract_txt:interactive in 435) [ClassicSimilarity], result of:
           0.27399844 = score(doc=435,freq=4.0), product of:
             0.31568632 = queryWeight, product of:
               2.7538154 = boost
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.020637099 = queryNorm
             0.86794525 = fieldWeight in 435, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
         0.12698188 = weight(abstract_txt:systems in 435) [ClassicSimilarity], result of:
           0.12698188 = score(doc=435,freq=4.0), product of:
             0.23819259 = queryWeight, product of:
               3.3828757 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.020637099 = queryNorm
             0.53310597 = fieldWeight in 435, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=435)
       0.48 = coord(12/25)
   

2. López-Ostenero, F.; Peinado, V.; Gonzalo, J.; Verdejo, F.: Interactive question answering : Is Cross-Language harder than monolingual searching? (2008) 0.23

   0.22565055 = sum of:
     0.22565055 = product of:
       0.705158 = sum of:
         0.059041336 = weight(abstract_txt:takes in 2023) [ClassicSimilarity], result of:
           0.059041336 = score(doc=2023,freq=1.0), product of:
             0.124884166 = queryWeight, product of:
               6.0514402 = idf(docFreq=282, maxDocs=44218)
               0.020637099 = queryNorm
             0.47276878 = fieldWeight in 2023, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0514402 = idf(docFreq=282, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.07916707 = weight(abstract_txt:answering in 2023) [ClassicSimilarity], result of:
           0.07916707 = score(doc=2023,freq=1.0), product of:
             0.15185636 = queryWeight, product of:
               1.1027138 = boost
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.020637099 = queryNorm
             0.5213286 = fieldWeight in 2023, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.6730065 = idf(docFreq=151, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.015881723 = weight(abstract_txt:this in 2023) [ClassicSimilarity], result of:
           0.015881723 = score(doc=2023,freq=2.0), product of:
             0.059570618 = queryWeight, product of:
               1.1962528 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.020637099 = queryNorm
             0.2666033 = fieldWeight in 2023, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.03319734 = weight(abstract_txt:using in 2023) [ClassicSimilarity], result of:
           0.03319734 = score(doc=2023,freq=1.0), product of:
             0.12270037 = queryWeight, product of:
               1.7168401 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.020637099 = queryNorm
             0.27055615 = fieldWeight in 2023, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.16217424 = weight(abstract_txt:cross in 2023) [ClassicSimilarity], result of:
           0.16217424 = score(doc=2023,freq=3.0), product of:
             0.21397388 = queryWeight, product of:
               1.8511503 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.020637099 = queryNorm
             0.75791603 = fieldWeight in 2023, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.07216041 = weight(abstract_txt:evaluation in 2023) [ClassicSimilarity], result of:
           0.07216041 = score(doc=2023,freq=1.0), product of:
             0.20589368 = queryWeight, product of:
               2.2239676 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.020637099 = queryNorm
             0.35047412 = fieldWeight in 2023, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.19374615 = weight(abstract_txt:interactive in 2023) [ClassicSimilarity], result of:
           0.19374615 = score(doc=2023,freq=2.0), product of:
             0.31568632 = queryWeight, product of:
               2.7538154 = boost
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.020637099 = queryNorm
             0.61372995 = fieldWeight in 2023, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
         0.08978976 = weight(abstract_txt:systems in 2023) [ClassicSimilarity], result of:
           0.08978976 = score(doc=2023,freq=2.0), product of:
             0.23819259 = queryWeight, product of:
               3.3828757 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.020637099 = queryNorm
             0.37696287 = fieldWeight in 2023, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.078125 = fieldNorm(doc=2023)
       0.32 = coord(8/25)
   

3. Syn, S.Y.; Spring, M.B.: Finding subject terms for classificatory metadata from user-generated social tags (2013) 0.21

   0.20963438 = sum of:
     0.20963438 = product of:
       0.65510744 = sum of:
         0.069802254 = weight(abstract_txt:judgments in 745) [ClassicSimilarity], result of:
           0.069802254 = score(doc=745,freq=1.0), product of:
             0.16202782 = queryWeight, product of:
               1.1390456 = boost
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.020637099 = queryNorm
             0.43080413 = fieldWeight in 745, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.892866 = idf(docFreq=121, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.008984059 = weight(abstract_txt:this in 745) [ClassicSimilarity], result of:
           0.008984059 = score(doc=745,freq=1.0), product of:
             0.059570618 = queryWeight, product of:
               1.1962528 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.020637099 = queryNorm
             0.1508136 = fieldWeight in 745, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.14806521 = weight(abstract_txt:measurements in 745) [ClassicSimilarity], result of:
           0.14806521 = score(doc=745,freq=2.0), product of:
             0.21230912 = queryWeight, product of:
               1.303859 = boost
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.020637099 = queryNorm
             0.6974039 = fieldWeight in 745, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.12474363 = weight(abstract_txt:remove in 745) [ClassicSimilarity], result of:
           0.12474363 = score(doc=745,freq=1.0), product of:
             0.23861013 = queryWeight, product of:
               1.3822632 = boost
               8.364683 = idf(docFreq=27, maxDocs=44218)
               0.020637099 = queryNorm
             0.5227927 = fieldWeight in 745, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.364683 = idf(docFreq=27, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.04599958 = weight(abstract_txt:using in 745) [ClassicSimilarity], result of:
           0.04599958 = score(doc=745,freq=3.0), product of:
             0.12270037 = queryWeight, product of:
               1.7168401 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.020637099 = queryNorm
             0.37489358 = fieldWeight in 745, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.07490507 = weight(abstract_txt:cross in 745) [ClassicSimilarity], result of:
           0.07490507 = score(doc=745,freq=1.0), product of:
             0.21397388 = queryWeight, product of:
               1.8511503 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.020637099 = queryNorm
             0.35006642 = fieldWeight in 745, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.094632015 = weight(abstract_txt:comparing in 745) [ClassicSimilarity], result of:
           0.094632015 = score(doc=745,freq=1.0), product of:
             0.25006065 = queryWeight, product of:
               2.00117 = boost
               6.0549803 = idf(docFreq=281, maxDocs=44218)
               0.020637099 = queryNorm
             0.37843627 = fieldWeight in 745, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0549803 = idf(docFreq=281, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
         0.087975636 = weight(abstract_txt:systems in 745) [ClassicSimilarity], result of:
           0.087975636 = score(doc=745,freq=3.0), product of:
             0.23819259 = queryWeight, product of:
               3.3828757 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.020637099 = queryNorm
             0.36934665 = fieldWeight in 745, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=745)
       0.32 = coord(8/25)
   

4. Petrelli, D.: On the role of user-centred evaluation in the advancement of interactive information retrieval (2008) 0.13

   0.13496478 = sum of:
     0.13496478 = product of:
       0.48201707 = sum of:
         0.06314713 = weight(abstract_txt:indeed in 2026) [ClassicSimilarity], result of:
           0.06314713 = score(doc=2026,freq=1.0), product of:
             0.15155806 = queryWeight, product of:
               1.1016302 = boost
               6.666449 = idf(docFreq=152, maxDocs=44218)
               0.020637099 = queryNorm
             0.41665307 = fieldWeight in 2026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.666449 = idf(docFreq=152, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
         0.012705378 = weight(abstract_txt:this in 2026) [ClassicSimilarity], result of:
           0.012705378 = score(doc=2026,freq=2.0), product of:
             0.059570618 = queryWeight, product of:
               1.1962528 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.020637099 = queryNorm
             0.21328263 = fieldWeight in 2026, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
         0.12973939 = weight(abstract_txt:cross in 2026) [ClassicSimilarity], result of:
           0.12973939 = score(doc=2026,freq=3.0), product of:
             0.21397388 = queryWeight, product of:
               1.8511503 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.020637099 = queryNorm
             0.60633284 = fieldWeight in 2026, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
         0.057728324 = weight(abstract_txt:evaluation in 2026) [ClassicSimilarity], result of:
           0.057728324 = score(doc=2026,freq=1.0), product of:
             0.20589368 = queryWeight, product of:
               2.2239676 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.020637099 = queryNorm
             0.2803793 = fieldWeight in 2026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
         0.058304716 = weight(abstract_txt:method in 2026) [ClassicSimilarity], result of:
           0.058304716 = score(doc=2026,freq=1.0), product of:
             0.2072619 = queryWeight, product of:
               2.2313447 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.020637099 = queryNorm
             0.28130937 = fieldWeight in 2026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
         0.109599374 = weight(abstract_txt:interactive in 2026) [ClassicSimilarity], result of:
           0.109599374 = score(doc=2026,freq=1.0), product of:
             0.31568632 = queryWeight, product of:
               2.7538154 = boost
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.020637099 = queryNorm
             0.3471781 = fieldWeight in 2026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
         0.050792757 = weight(abstract_txt:systems in 2026) [ClassicSimilarity], result of:
           0.050792757 = score(doc=2026,freq=1.0), product of:
             0.23819259 = queryWeight, product of:
               3.3828757 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.020637099 = queryNorm
             0.2132424 = fieldWeight in 2026, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=2026)
       0.28 = coord(7/25)
   

5. Zhou, D.; Lawless, S.; Wu, X.; Zhao, W.; Liu, J.: ¬A study of user profile representation for personalized cross-language information retrieval (2016) 0.13

   0.13311519 = sum of:
     0.13311519 = product of:
       0.47541136 = sum of:
         0.012705378 = weight(abstract_txt:this in 3167) [ClassicSimilarity], result of:
           0.012705378 = score(doc=3167,freq=2.0), product of:
             0.059570618 = queryWeight, product of:
               1.1962528 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.020637099 = queryNorm
             0.21328263 = fieldWeight in 3167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
         0.02655787 = weight(abstract_txt:using in 3167) [ClassicSimilarity], result of:
           0.02655787 = score(doc=3167,freq=1.0), product of:
             0.12270037 = queryWeight, product of:
               1.7168401 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.020637099 = queryNorm
             0.21644491 = fieldWeight in 3167, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
         0.12973939 = weight(abstract_txt:cross in 3167) [ClassicSimilarity], result of:
           0.12973939 = score(doc=3167,freq=3.0), product of:
             0.21397388 = queryWeight, product of:
               1.8511503 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.020637099 = queryNorm
             0.60633284 = fieldWeight in 3167, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
         0.094632015 = weight(abstract_txt:comparing in 3167) [ClassicSimilarity], result of:
           0.094632015 = score(doc=3167,freq=1.0), product of:
             0.25006065 = queryWeight, product of:
               2.00117 = boost
               6.0549803 = idf(docFreq=281, maxDocs=44218)
               0.020637099 = queryNorm
             0.37843627 = fieldWeight in 3167, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0549803 = idf(docFreq=281, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
         0.081640184 = weight(abstract_txt:evaluation in 3167) [ClassicSimilarity], result of:
           0.081640184 = score(doc=3167,freq=2.0), product of:
             0.20589368 = queryWeight, product of:
               2.2239676 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.020637099 = queryNorm
             0.3965162 = fieldWeight in 3167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
         0.058304716 = weight(abstract_txt:method in 3167) [ClassicSimilarity], result of:
           0.058304716 = score(doc=3167,freq=1.0), product of:
             0.2072619 = queryWeight, product of:
               2.2313447 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.020637099 = queryNorm
             0.28130937 = fieldWeight in 3167, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
         0.07183181 = weight(abstract_txt:systems in 3167) [ClassicSimilarity], result of:
           0.07183181 = score(doc=3167,freq=2.0), product of:
             0.23819259 = queryWeight, product of:
               3.3828757 = boost
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.020637099 = queryNorm
             0.3015703 = fieldWeight in 3167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4118783 = idf(docFreq=3963, maxDocs=44218)
               0.0625 = fieldNorm(doc=3167)
       0.28 = coord(7/25)