Document (#30170)

Author

Ruthven, T.
Lalmas, M.
Rijsbergen, K.van

Title

Incorporating user research behavior into relevance feedback

Source

Journal of the American Society for Information Science and technology. 54(2003) no.6, S.528-548

Year

2003

Abstract

Ruthven, Mounia, and van Rijsbergen rank and select terms for query expansion using information gathered on searcher evaluation behavior. Using the TREC Financial Times and Los Angeles Times collections and search topics from TREC-6 placed in simulated work situations, six student subjects each preformed three searches on an experimental system and three on a control system with instructions to search by natural language expression in any way they found comfortable. Searching was analyzed for behavior differences between experimental and control situations, and for effectiveness and perceptions. In three experiments paired t-tests were the analysis tool with controls being a no relevance feedback system, a standard ranking for automatic expansion system, and a standard ranking for interactive expansion while the experimental systems based ranking upon user information on temporal relevance and partial relevance. Two further experiments compare using user behavior (number assessed relevant and similarity of relevant documents) to choose a query expansion technique against a non-selective technique and finally the effect of providing the user with knowledge of the process. When partial relevance data and time of assessment data are incorporated in term ranking more relevant documents were recovered in fewer iterations, however retrieval effectiveness overall was not improved. The subjects, none-the-less, rated the suggested terms as more useful and used them more heavily. Explanations of what the feedback techniques were doing led to higher use of the techniques.

Theme

Retrievalalgorithmen

Similar documents (author)

1. Ruthven, I.; Lalmas, M.; Rijsbergen, K. van: Combining and selecting characteristics of information use (2002) 5.44

   5.4365253 = sum of:
     5.4365253 = sum of:
       1.7226502 = weight(author_txt:ruthven in 5208) [ClassicSimilarity], result of:
         1.7226502 = score(doc=5208,freq=1.0), product of:
           0.5580913 = queryWeight, product of:
             8.231152 = idf(docFreq=31, maxDocs=44218)
             0.06780233 = queryNorm
           3.0866818 = fieldWeight in 5208, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.231152 = idf(docFreq=31, maxDocs=44218)
             0.375 = fieldNorm(doc=5208)
       1.8315387 = weight(author_txt:rijsbergen in 5208) [ClassicSimilarity], result of:
         1.8315387 = score(doc=5208,freq=1.0), product of:
           0.58136815 = queryWeight, product of:
             1.020641 = boost
             8.401051 = idf(docFreq=26, maxDocs=44218)
             0.06780233 = queryNorm
           3.150394 = fieldWeight in 5208, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.401051 = idf(docFreq=26, maxDocs=44218)
             0.375 = fieldNorm(doc=5208)
       1.8823364 = weight(author_txt:lalmas in 5208) [ClassicSimilarity], result of:
         1.8823364 = score(doc=5208,freq=1.0), product of:
           0.59206855 = queryWeight, product of:
             1.0299909 = boost
             8.478011 = idf(docFreq=24, maxDocs=44218)
             0.06780233 = queryNorm
           3.179254 = fieldWeight in 5208, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.478011 = idf(docFreq=24, maxDocs=44218)
             0.375 = fieldNorm(doc=5208)
   

2. Lalmas, M.; Ruthven, I.: ¬A model for structured document retrieval : empirical investigations (1997) 3.20

   3.2044325 = sum of:
     3.2044325 = product of:
       4.8066487 = sum of:
         2.296867 = weight(author_txt:ruthven in 727) [ClassicSimilarity], result of:
           2.296867 = score(doc=727,freq=1.0), product of:
             0.5580913 = queryWeight, product of:
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.06780233 = queryNorm
             4.115576 = fieldWeight in 727, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.5 = fieldNorm(doc=727)
         2.5097818 = weight(author_txt:lalmas in 727) [ClassicSimilarity], result of:
           2.5097818 = score(doc=727,freq=1.0), product of:
             0.59206855 = queryWeight, product of:
               1.0299909 = boost
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.06780233 = queryNorm
             4.2390056 = fieldWeight in 727, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.5 = fieldNorm(doc=727)
       0.6666667 = coord(2/3)
   

3. Lalmas, M.; Ruthven, I.: Representing and retrieving structured documents using the Dempster-Shafer theory of evidence : modelling and evaluation (1998) 3.20

   3.2044325 = sum of:
     3.2044325 = product of:
       4.8066487 = sum of:
         2.296867 = weight(author_txt:ruthven in 1076) [ClassicSimilarity], result of:
           2.296867 = score(doc=1076,freq=1.0), product of:
             0.5580913 = queryWeight, product of:
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.06780233 = queryNorm
             4.115576 = fieldWeight in 1076, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.5 = fieldNorm(doc=1076)
         2.5097818 = weight(author_txt:lalmas in 1076) [ClassicSimilarity], result of:
           2.5097818 = score(doc=1076,freq=1.0), product of:
             0.59206855 = queryWeight, product of:
               1.0299909 = boost
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.06780233 = queryNorm
             4.2390056 = fieldWeight in 1076, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.5 = fieldNorm(doc=1076)
       0.6666667 = coord(2/3)
   

4. Ruthven, I.; Lalmas, M.: Selective relevance feedback using term characteristics (1999) 3.20

   3.2044325 = sum of:
     3.2044325 = product of:
       4.8066487 = sum of:
         2.296867 = weight(author_txt:ruthven in 3824) [ClassicSimilarity], result of:
           2.296867 = score(doc=3824,freq=1.0), product of:
             0.5580913 = queryWeight, product of:
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.06780233 = queryNorm
             4.115576 = fieldWeight in 3824, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.231152 = idf(docFreq=31, maxDocs=44218)
               0.5 = fieldNorm(doc=3824)
         2.5097818 = weight(author_txt:lalmas in 3824) [ClassicSimilarity], result of:
           2.5097818 = score(doc=3824,freq=1.0), product of:
             0.59206855 = queryWeight, product of:
               1.0299909 = boost
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.06780233 = queryNorm
             4.2390056 = fieldWeight in 3824, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.5 = fieldNorm(doc=3824)
       0.6666667 = coord(2/3)
   

5. Rijsbergen, C.J. van; Lalmas, M.: Information calculus for information retrieval (1996) 2.89

   2.8885696 = sum of:
     2.8885696 = product of:
       4.3328543 = sum of:
         2.136795 = weight(author_txt:rijsbergen in 4201) [ClassicSimilarity], result of:
           2.136795 = score(doc=4201,freq=1.0), product of:
             0.58136815 = queryWeight, product of:
               1.020641 = boost
               8.401051 = idf(docFreq=26, maxDocs=44218)
               0.06780233 = queryNorm
             3.6754596 = fieldWeight in 4201, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.401051 = idf(docFreq=26, maxDocs=44218)
               0.4375 = fieldNorm(doc=4201)
         2.1960592 = weight(author_txt:lalmas in 4201) [ClassicSimilarity], result of:
           2.1960592 = score(doc=4201,freq=1.0), product of:
             0.59206855 = queryWeight, product of:
               1.0299909 = boost
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.06780233 = queryNorm
             3.7091298 = fieldWeight in 4201, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.478011 = idf(docFreq=24, maxDocs=44218)
               0.4375 = fieldNorm(doc=4201)
       0.6666667 = coord(2/3)
   

Similar documents (content)

1. Vakkari, P.; Sormunen, E.: ¬The influence of relevance levels an the effectiveness of interactive information retrieval (2004) 0.65

   0.6503884 = sum of:
     0.6503884 = product of:
       1.3549759 = sum of:
         0.07314995 = weight(abstract_txt:query in 2884) [ClassicSimilarity], result of:
           0.07314995 = score(doc=2884,freq=3.0), product of:
             0.11371708 = queryWeight, product of:
               1.0060412 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.023777831 = queryNorm
             0.6432626 = fieldWeight in 2884, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.05209985 = weight(abstract_txt:effectiveness in 2884) [ClassicSimilarity], result of:
           0.05209985 = score(doc=2884,freq=1.0), product of:
             0.130802 = queryWeight, product of:
               1.0789719 = boost
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.023777831 = queryNorm
             0.39831078 = fieldWeight in 2884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.024492532 = weight(abstract_txt:using in 2884) [ClassicSimilarity], result of:
           0.024492532 = score(doc=2884,freq=1.0), product of:
             0.09052661 = queryWeight, product of:
               1.0993518 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.023777831 = queryNorm
             0.27055615 = fieldWeight in 2884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.08420519 = weight(abstract_txt:experiments in 2884) [ClassicSimilarity], result of:
           0.08420519 = score(doc=2884,freq=2.0), product of:
             0.14297915 = queryWeight, product of:
               1.1280783 = boost
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.023777831 = queryNorm
             0.58893335 = fieldWeight in 2884, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.06266846 = weight(abstract_txt:subjects in 2884) [ClassicSimilarity], result of:
           0.06266846 = score(doc=2884,freq=1.0), product of:
             0.14794129 = queryWeight, product of:
               1.1474866 = boost
               5.4221253 = idf(docFreq=530, maxDocs=44218)
               0.023777831 = queryNorm
             0.42360353 = fieldWeight in 2884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4221253 = idf(docFreq=530, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.02915093 = weight(abstract_txt:were in 2884) [ClassicSimilarity], result of:
           0.02915093 = score(doc=2884,freq=1.0), product of:
             0.10166905 = queryWeight, product of:
               1.1650456 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.023777831 = queryNorm
             0.28672373 = fieldWeight in 2884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.16719243 = weight(abstract_txt:trec in 2884) [ClassicSimilarity], result of:
           0.16719243 = score(doc=2884,freq=2.0), product of:
             0.22586963 = queryWeight, product of:
               1.417855 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.023777831 = queryNorm
             0.7402165 = fieldWeight in 2884, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.101741195 = weight(abstract_txt:relevant in 2884) [ClassicSimilarity], result of:
           0.101741195 = score(doc=2884,freq=3.0), product of:
             0.16219768 = queryWeight, product of:
               1.4715359 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.023777831 = queryNorm
             0.62726665 = fieldWeight in 2884, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.039297774 = weight(abstract_txt:user in 2884) [ClassicSimilarity], result of:
           0.039297774 = score(doc=2884,freq=1.0), product of:
             0.1365564 = queryWeight, product of:
               1.5590999 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.023777831 = queryNorm
             0.2877769 = fieldWeight in 2884, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.17516688 = weight(abstract_txt:feedback in 2884) [ClassicSimilarity], result of:
           0.17516688 = score(doc=2884,freq=2.0), product of:
             0.2667136 = queryWeight, product of:
               1.8869965 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.023777831 = queryNorm
             0.6567602 = fieldWeight in 2884, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.31001318 = weight(abstract_txt:expansion in 2884) [ClassicSimilarity], result of:
           0.31001318 = score(doc=2884,freq=3.0), product of:
             0.37521464 = queryWeight, product of:
               2.5843885 = boost
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.023777831 = queryNorm
             0.82622886 = fieldWeight in 2884, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
         0.23579763 = weight(abstract_txt:relevance in 2884) [ClassicSimilarity], result of:
           0.23579763 = score(doc=2884,freq=4.0), product of:
             0.30599174 = queryWeight, product of:
               2.6093225 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023777831 = queryNorm
             0.7706013 = fieldWeight in 2884, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.078125 = fieldNorm(doc=2884)
       0.48 = coord(12/25)
   

2. Efthimiadis, E.N.: Interactive query expansion : a user-based evaluation in a relevance feedback environment (2000) 0.52

   0.52204704 = sum of:
     0.52204704 = product of:
       1.0875981 = sum of:
         0.121892184 = weight(abstract_txt:query in 5701) [ClassicSimilarity], result of:
           0.121892184 = score(doc=5701,freq=17.0), product of:
             0.11371708 = queryWeight, product of:
               1.0060412 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.023777831 = queryNorm
             1.0718899 = fieldWeight in 5701, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.05157622 = weight(abstract_txt:effectiveness in 5701) [ClassicSimilarity], result of:
           0.05157622 = score(doc=5701,freq=2.0), product of:
             0.130802 = queryWeight, product of:
               1.0789719 = boost
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.023777831 = queryNorm
             0.39430755 = fieldWeight in 5701, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.098378 = idf(docFreq=733, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.016254224 = weight(abstract_txt:more in 5701) [ClassicSimilarity], result of:
           0.016254224 = score(doc=5701,freq=1.0), product of:
             0.087364025 = queryWeight, product of:
               1.0799779 = boost
               3.402088 = idf(docFreq=4002, maxDocs=44218)
               0.023777831 = queryNorm
             0.18605168 = fieldWeight in 5701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.402088 = idf(docFreq=4002, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.017144771 = weight(abstract_txt:using in 5701) [ClassicSimilarity], result of:
           0.017144771 = score(doc=5701,freq=1.0), product of:
             0.09052661 = queryWeight, product of:
               1.0993518 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.023777831 = queryNorm
             0.18938929 = fieldWeight in 5701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.045628425 = weight(abstract_txt:were in 5701) [ClassicSimilarity], result of:
           0.045628425 = score(doc=5701,freq=5.0), product of:
             0.10166905 = queryWeight, product of:
               1.1650456 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.023777831 = queryNorm
             0.44879365 = fieldWeight in 5701, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.03555273 = weight(abstract_txt:three in 5701) [ClassicSimilarity], result of:
           0.03555273 = score(doc=5701,freq=1.0), product of:
             0.14720993 = queryWeight, product of:
               1.4019002 = boost
               4.41619 = idf(docFreq=1451, maxDocs=44218)
               0.023777831 = queryNorm
             0.24151039 = fieldWeight in 5701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.41619 = idf(docFreq=1451, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.021108154 = weight(abstract_txt:system in 5701) [ClassicSimilarity], result of:
           0.021108154 = score(doc=5701,freq=1.0), product of:
             0.11445503 = queryWeight, product of:
               1.427366 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.023777831 = queryNorm
             0.18442312 = fieldWeight in 5701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.058149938 = weight(abstract_txt:relevant in 5701) [ClassicSimilarity], result of:
           0.058149938 = score(doc=5701,freq=2.0), product of:
             0.16219768 = queryWeight, product of:
               1.4715359 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.023777831 = queryNorm
             0.35851276 = fieldWeight in 5701, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.03890281 = weight(abstract_txt:user in 5701) [ClassicSimilarity], result of:
           0.03890281 = score(doc=5701,freq=2.0), product of:
             0.1365564 = queryWeight, product of:
               1.5590999 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.023777831 = queryNorm
             0.28488457 = fieldWeight in 5701, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.086703174 = weight(abstract_txt:feedback in 5701) [ClassicSimilarity], result of:
           0.086703174 = score(doc=5701,freq=1.0), product of:
             0.2667136 = queryWeight, product of:
               1.8869965 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.023777831 = queryNorm
             0.3250797 = fieldWeight in 5701, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.4517407 = weight(abstract_txt:expansion in 5701) [ClassicSimilarity], result of:
           0.4517407 = score(doc=5701,freq=13.0), product of:
             0.37521464 = queryWeight, product of:
               2.5843885 = boost
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.023777831 = queryNorm
             1.2039528 = fieldWeight in 5701, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
         0.14294471 = weight(abstract_txt:relevance in 5701) [ClassicSimilarity], result of:
           0.14294471 = score(doc=5701,freq=3.0), product of:
             0.30599174 = queryWeight, product of:
               2.6093225 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023777831 = queryNorm
             0.46715218 = fieldWeight in 5701, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0546875 = fieldNorm(doc=5701)
       0.48 = coord(12/25)
   

3. Robertson, S.E.: OKAPI at TREC-3 (1995) 0.50

   0.496016 = sum of:
     0.496016 = product of:
       1.24004 = sum of:
         0.05067977 = weight(abstract_txt:query in 5694) [ClassicSimilarity], result of:
           0.05067977 = score(doc=5694,freq=1.0), product of:
             0.11371708 = queryWeight, product of:
               1.0060412 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.023777831 = queryNorm
             0.44566542 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.029391035 = weight(abstract_txt:using in 5694) [ClassicSimilarity], result of:
           0.029391035 = score(doc=5694,freq=1.0), product of:
             0.09052661 = queryWeight, product of:
               1.0993518 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.023777831 = queryNorm
             0.32466736 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.07145048 = weight(abstract_txt:experiments in 5694) [ClassicSimilarity], result of:
           0.07145048 = score(doc=5694,freq=1.0), product of:
             0.14297915 = queryWeight, product of:
               1.1280783 = boost
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.023777831 = queryNorm
             0.49972656 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.24572167 = weight(abstract_txt:trec in 5694) [ClassicSimilarity], result of:
           0.24572167 = score(doc=5694,freq=3.0), product of:
             0.22586963 = queryWeight, product of:
               1.417855 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.023777831 = queryNorm
             1.0878916 = fieldWeight in 5694, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.036185406 = weight(abstract_txt:system in 5694) [ClassicSimilarity], result of:
           0.036185406 = score(doc=5694,freq=1.0), product of:
             0.11445503 = queryWeight, product of:
               1.427366 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.023777831 = queryNorm
             0.3161539 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.047157332 = weight(abstract_txt:user in 5694) [ClassicSimilarity], result of:
           0.047157332 = score(doc=5694,freq=1.0), product of:
             0.1365564 = queryWeight, product of:
               1.5590999 = boost
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.023777831 = queryNorm
             0.34533226 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6835442 = idf(docFreq=3020, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.148634 = weight(abstract_txt:feedback in 5694) [ClassicSimilarity], result of:
           0.148634 = score(doc=5694,freq=1.0), product of:
             0.2667136 = queryWeight, product of:
               1.8869965 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.023777831 = queryNorm
             0.55727947 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.16559206 = weight(abstract_txt:ranking in 5694) [ClassicSimilarity], result of:
           0.16559206 = score(doc=5694,freq=1.0), product of:
             0.3154803 = queryWeight, product of:
               2.3697581 = boost
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.023777831 = queryNorm
             0.52488875 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.30374962 = weight(abstract_txt:expansion in 5694) [ClassicSimilarity], result of:
           0.30374962 = score(doc=5694,freq=2.0), product of:
             0.37521464 = queryWeight, product of:
               2.5843885 = boost
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.023777831 = queryNorm
             0.8095356 = fieldWeight in 5694, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
         0.14147857 = weight(abstract_txt:relevance in 5694) [ClassicSimilarity], result of:
           0.14147857 = score(doc=5694,freq=1.0), product of:
             0.30599174 = queryWeight, product of:
               2.6093225 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023777831 = queryNorm
             0.46236074 = fieldWeight in 5694, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.09375 = fieldNorm(doc=5694)
       0.4 = coord(10/25)
   

4. Xu, B.; Lin, H.; Lin, Y.: Assessment of learning to rank methods for query expansion (2016) 0.44

   0.44050822 = sum of:
     0.44050822 = product of:
       1.1012706 = sum of:
         0.04778135 = weight(abstract_txt:query in 2929) [ClassicSimilarity], result of:
           0.04778135 = score(doc=2929,freq=2.0), product of:
             0.11371708 = queryWeight, product of:
               1.0060412 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.023777831 = queryNorm
             0.4201774 = fieldWeight in 2929, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.019594025 = weight(abstract_txt:using in 2929) [ClassicSimilarity], result of:
           0.019594025 = score(doc=2929,freq=1.0), product of:
             0.09052661 = queryWeight, product of:
               1.0993518 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.023777831 = queryNorm
             0.21644491 = fieldWeight in 2929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.04063169 = weight(abstract_txt:three in 2929) [ClassicSimilarity], result of:
           0.04063169 = score(doc=2929,freq=1.0), product of:
             0.14720993 = queryWeight, product of:
               1.4019002 = boost
               4.41619 = idf(docFreq=1451, maxDocs=44218)
               0.023777831 = queryNorm
             0.27601188 = fieldWeight in 2929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.41619 = idf(docFreq=1451, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.09457832 = weight(abstract_txt:trec in 2929) [ClassicSimilarity], result of:
           0.09457832 = score(doc=2929,freq=1.0), product of:
             0.22586963 = queryWeight, product of:
               1.417855 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.023777831 = queryNorm
             0.4187297 = fieldWeight in 2929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.046992246 = weight(abstract_txt:relevant in 2929) [ClassicSimilarity], result of:
           0.046992246 = score(doc=2929,freq=1.0), product of:
             0.16219768 = queryWeight, product of:
               1.4715359 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.023777831 = queryNorm
             0.28972206 = fieldWeight in 2929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.07420023 = weight(abstract_txt:experimental in 2929) [ClassicSimilarity], result of:
           0.07420023 = score(doc=2929,freq=1.0), product of:
             0.21993652 = queryWeight, product of:
               1.7135518 = boost
               5.397938 = idf(docFreq=543, maxDocs=44218)
               0.023777831 = queryNorm
             0.3373711 = fieldWeight in 2929, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.397938 = idf(docFreq=543, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.17162777 = weight(abstract_txt:feedback in 2929) [ClassicSimilarity], result of:
           0.17162777 = score(doc=2929,freq=3.0), product of:
             0.2667136 = queryWeight, product of:
               1.8869965 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.023777831 = queryNorm
             0.6434909 = fieldWeight in 2929, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.15612167 = weight(abstract_txt:ranking in 2929) [ClassicSimilarity], result of:
           0.15612167 = score(doc=2929,freq=2.0), product of:
             0.3154803 = queryWeight, product of:
               2.3697581 = boost
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.023777831 = queryNorm
             0.49486983 = fieldWeight in 2929, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.598813 = idf(docFreq=444, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.2863779 = weight(abstract_txt:expansion in 2929) [ClassicSimilarity], result of:
           0.2863779 = score(doc=2929,freq=4.0), product of:
             0.37521464 = queryWeight, product of:
               2.5843885 = boost
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.023777831 = queryNorm
             0.76323754 = fieldWeight in 2929, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
         0.16336538 = weight(abstract_txt:relevance in 2929) [ClassicSimilarity], result of:
           0.16336538 = score(doc=2929,freq=3.0), product of:
             0.30599174 = queryWeight, product of:
               2.6093225 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023777831 = queryNorm
             0.5338882 = fieldWeight in 2929, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.0625 = fieldNorm(doc=2929)
       0.4 = coord(10/25)
   

5. Robertson, S.E.; Walker, S.; Hancock-Beaulieu, M.M.: Large test collection experiments of an operational, interactive system : OKAPI at TREC (1995) 0.40

   0.3972082 = sum of:
     0.3972082 = product of:
       1.1033561 = sum of:
         0.071672015 = weight(abstract_txt:query in 6964) [ClassicSimilarity], result of:
           0.071672015 = score(doc=6964,freq=2.0), product of:
             0.11371708 = queryWeight, product of:
               1.0060412 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.023777831 = queryNorm
             0.6302661 = fieldWeight in 6964, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.027864384 = weight(abstract_txt:more in 6964) [ClassicSimilarity], result of:
           0.027864384 = score(doc=6964,freq=1.0), product of:
             0.087364025 = queryWeight, product of:
               1.0799779 = boost
               3.402088 = idf(docFreq=4002, maxDocs=44218)
               0.023777831 = queryNorm
             0.31894574 = fieldWeight in 6964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.402088 = idf(docFreq=4002, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.07145048 = weight(abstract_txt:experiments in 6964) [ClassicSimilarity], result of:
           0.07145048 = score(doc=6964,freq=1.0), product of:
             0.14297915 = queryWeight, product of:
               1.1280783 = boost
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.023777831 = queryNorm
             0.49972656 = fieldWeight in 6964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3304167 = idf(docFreq=581, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.20063092 = weight(abstract_txt:trec in 6964) [ClassicSimilarity], result of:
           0.20063092 = score(doc=6964,freq=2.0), product of:
             0.22586963 = queryWeight, product of:
               1.417855 = boost
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.023777831 = queryNorm
             0.8882598 = fieldWeight in 6964, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.699675 = idf(docFreq=147, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.036185406 = weight(abstract_txt:system in 6964) [ClassicSimilarity], result of:
           0.036185406 = score(doc=6964,freq=1.0), product of:
             0.11445503 = queryWeight, product of:
               1.427366 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.023777831 = queryNorm
             0.3161539 = fieldWeight in 6964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.07048836 = weight(abstract_txt:relevant in 6964) [ClassicSimilarity], result of:
           0.07048836 = score(doc=6964,freq=1.0), product of:
             0.16219768 = queryWeight, product of:
               1.4715359 = boost
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.023777831 = queryNorm
             0.43458307 = fieldWeight in 6964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.635553 = idf(docFreq=1165, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.21020024 = weight(abstract_txt:feedback in 6964) [ClassicSimilarity], result of:
           0.21020024 = score(doc=6964,freq=2.0), product of:
             0.2667136 = queryWeight, product of:
               1.8869965 = boost
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.023777831 = queryNorm
             0.7881122 = fieldWeight in 6964, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9443145 = idf(docFreq=314, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.21478343 = weight(abstract_txt:expansion in 6964) [ClassicSimilarity], result of:
           0.21478343 = score(doc=6964,freq=1.0), product of:
             0.37521464 = queryWeight, product of:
               2.5843885 = boost
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.023777831 = queryNorm
             0.57242817 = fieldWeight in 6964, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1059003 = idf(docFreq=267, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
         0.20008092 = weight(abstract_txt:relevance in 6964) [ClassicSimilarity], result of:
           0.20008092 = score(doc=6964,freq=2.0), product of:
             0.30599174 = queryWeight, product of:
               2.6093225 = boost
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.023777831 = queryNorm
             0.65387684 = fieldWeight in 6964, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.931848 = idf(docFreq=866, maxDocs=44218)
               0.09375 = fieldNorm(doc=6964)
       0.36 = coord(9/25)