Document (#30168)

Author

Ruthven, T.
Lalmas, M.
Rijsbergen, K.van

Title

Incorporating user research behavior into relevance feedback

Source

Journal of the American Society for Information Science and technology. 54(2003) no.6, S.528-548

Year

2003

Abstract

Ruthven, Mounia, and van Rijsbergen rank and select terms for query expansion using information gathered on searcher evaluation behavior. Using the TREC Financial Times and Los Angeles Times collections and search topics from TREC-6 placed in simulated work situations, six student subjects each preformed three searches on an experimental system and three on a control system with instructions to search by natural language expression in any way they found comfortable. Searching was analyzed for behavior differences between experimental and control situations, and for effectiveness and perceptions. In three experiments paired t-tests were the analysis tool with controls being a no relevance feedback system, a standard ranking for automatic expansion system, and a standard ranking for interactive expansion while the experimental systems based ranking upon user information on temporal relevance and partial relevance. Two further experiments compare using user behavior (number assessed relevant and similarity of relevant documents) to choose a query expansion technique against a non-selective technique and finally the effect of providing the user with knowledge of the process. When partial relevance data and time of assessment data are incorporated in term ranking more relevant documents were recovered in fewer iterations, however retrieval effectiveness overall was not improved. The subjects, none-the-less, rated the suggested terms as more useful and used them more heavily. Explanations of what the feedback techniques were doing led to higher use of the techniques.

Theme

Retrievalalgorithmen

Similar documents (author)

1. Ruthven, I.; Lalmas, M.; Rijsbergen, K. van: Combining and selecting characteristics of information use (2002) 5.43

   5.4335995 = sum of:
     5.4335995 = sum of:
       1.7351995 = weight(author_txt:ruthven in 206) [ClassicSimilarity], result of:
         1.7351995 = score(doc=206,freq=1.0), product of:
           0.5610211 = queryWeight, product of:
             8.247815 = idf(docFreq=30, maxDocs=43556)
             0.06802057 = queryNorm
           3.0929308 = fieldWeight in 206, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.247815 = idf(docFreq=30, maxDocs=43556)
             0.375 = fieldNorm(doc=206)
       1.8238617 = weight(author_txt:rijsbergen in 206) [ClassicSimilarity], result of:
         1.8238617 = score(doc=206,freq=1.0), product of:
           0.5799727 = queryWeight, product of:
             1.01675 = boost
             8.385966 = idf(docFreq=26, maxDocs=43556)
             0.06802057 = queryNorm
           3.1447372 = fieldWeight in 206, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.385966 = idf(docFreq=26, maxDocs=43556)
             0.375 = fieldNorm(doc=206)
       1.8745384 = weight(author_txt:lalmas in 206) [ClassicSimilarity], result of:
         1.8745384 = score(doc=206,freq=1.0), product of:
           0.5906667 = queryWeight, product of:
             1.0260811 = boost
             8.462927 = idf(docFreq=24, maxDocs=43556)
             0.06802057 = queryNorm
           3.1735976 = fieldWeight in 206, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.462927 = idf(docFreq=24, maxDocs=43556)
             0.375 = fieldNorm(doc=206)
   

2. Lalmas, M.; Ruthven, I.: ¬A model for structured document retrieval : empirical investigations (1997) 3.21

   3.2086558 = sum of:
     3.2086558 = product of:
       4.8129835 = sum of:
         2.313599 = weight(author_txt:ruthven in 1725) [ClassicSimilarity], result of:
           2.313599 = score(doc=1725,freq=1.0), product of:
             0.5610211 = queryWeight, product of:
               8.247815 = idf(docFreq=30, maxDocs=43556)
               0.06802057 = queryNorm
             4.1239076 = fieldWeight in 1725, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.247815 = idf(docFreq=30, maxDocs=43556)
               0.5 = fieldNorm(doc=1725)
         2.4993846 = weight(author_txt:lalmas in 1725) [ClassicSimilarity], result of:
           2.4993846 = score(doc=1725,freq=1.0), product of:
             0.5906667 = queryWeight, product of:
               1.0260811 = boost
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.06802057 = queryNorm
             4.2314634 = fieldWeight in 1725, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.5 = fieldNorm(doc=1725)
       0.6666667 = coord(2/3)
   

3. Lalmas, M.; Ruthven, I.: Representing and retrieving structured documents using the Dempster-Shafer theory of evidence : modelling and evaluation (1998) 3.21

   3.2086558 = sum of:
     3.2086558 = product of:
       4.8129835 = sum of:
         2.313599 = weight(author_txt:ruthven in 2074) [ClassicSimilarity], result of:
           2.313599 = score(doc=2074,freq=1.0), product of:
             0.5610211 = queryWeight, product of:
               8.247815 = idf(docFreq=30, maxDocs=43556)
               0.06802057 = queryNorm
             4.1239076 = fieldWeight in 2074, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.247815 = idf(docFreq=30, maxDocs=43556)
               0.5 = fieldNorm(doc=2074)
         2.4993846 = weight(author_txt:lalmas in 2074) [ClassicSimilarity], result of:
           2.4993846 = score(doc=2074,freq=1.0), product of:
             0.5906667 = queryWeight, product of:
               1.0260811 = boost
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.06802057 = queryNorm
             4.2314634 = fieldWeight in 2074, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.5 = fieldNorm(doc=2074)
       0.6666667 = coord(2/3)
   

4. Ruthven, I.; Lalmas, M.: Selective relevance feedback using term characteristics (1999) 3.21

   3.2086558 = sum of:
     3.2086558 = product of:
       4.8129835 = sum of:
         2.313599 = weight(author_txt:ruthven in 4822) [ClassicSimilarity], result of:
           2.313599 = score(doc=4822,freq=1.0), product of:
             0.5610211 = queryWeight, product of:
               8.247815 = idf(docFreq=30, maxDocs=43556)
               0.06802057 = queryNorm
             4.1239076 = fieldWeight in 4822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.247815 = idf(docFreq=30, maxDocs=43556)
               0.5 = fieldNorm(doc=4822)
         2.4993846 = weight(author_txt:lalmas in 4822) [ClassicSimilarity], result of:
           2.4993846 = score(doc=4822,freq=1.0), product of:
             0.5906667 = queryWeight, product of:
               1.0260811 = boost
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.06802057 = queryNorm
             4.2314634 = fieldWeight in 4822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.5 = fieldNorm(doc=4822)
       0.6666667 = coord(2/3)
   

5. Rijsbergen, C.J. van; Lalmas, M.: Information calculus for information retrieval (1996) 2.88

   2.8765335 = sum of:
     2.8765335 = product of:
       4.3148003 = sum of:
         2.1278386 = weight(author_txt:rijsbergen in 4267) [ClassicSimilarity], result of:
           2.1278386 = score(doc=4267,freq=1.0), product of:
             0.5799727 = queryWeight, product of:
               1.01675 = boost
               8.385966 = idf(docFreq=26, maxDocs=43556)
               0.06802057 = queryNorm
             3.6688602 = fieldWeight in 4267, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.385966 = idf(docFreq=26, maxDocs=43556)
               0.4375 = fieldNorm(doc=4267)
         2.1869614 = weight(author_txt:lalmas in 4267) [ClassicSimilarity], result of:
           2.1869614 = score(doc=4267,freq=1.0), product of:
             0.5906667 = queryWeight, product of:
               1.0260811 = boost
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.06802057 = queryNorm
             3.7025304 = fieldWeight in 4267, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.462927 = idf(docFreq=24, maxDocs=43556)
               0.4375 = fieldNorm(doc=4267)
       0.6666667 = coord(2/3)
   

Similar documents (content)

1. Vakkari, P.; Sormunen, E.: ¬The influence of relevance levels an the effectiveness of interactive information retrieval (2004) 0.65

   0.64996654 = sum of:
     0.64996654 = product of:
       1.354097 = sum of:
         0.07247312 = weight(abstract_txt:query in 3882) [ClassicSimilarity], result of:
           0.07247312 = score(doc=3882,freq=3.0), product of:
             0.11293135 = queryWeight, product of:
               1.0036594 = boost
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.023725552 = queryNorm
             0.641745 = fieldWeight in 3882, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.05236946 = weight(abstract_txt:effectiveness in 3882) [ClassicSimilarity], result of:
           0.05236946 = score(doc=3882,freq=1.0), product of:
             0.13115625 = queryWeight, product of:
               1.0816172 = boost
               5.11092 = idf(docFreq=713, maxDocs=43556)
               0.023725552 = queryNorm
             0.39929062 = fieldWeight in 3882, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.11092 = idf(docFreq=713, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.024608582 = weight(abstract_txt:using in 3882) [ClassicSimilarity], result of:
           0.024608582 = score(doc=3882,freq=1.0), product of:
             0.0907455 = queryWeight, product of:
               1.1018876 = boost
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.023725552 = queryNorm
             0.2711824 = fieldWeight in 3882, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.08454186 = weight(abstract_txt:experiments in 3882) [ClassicSimilarity], result of:
           0.08454186 = score(doc=3882,freq=2.0), product of:
             0.14325446 = queryWeight, product of:
               1.1304027 = boost
               5.3414435 = idf(docFreq=566, maxDocs=43556)
               0.023725552 = queryNorm
             0.59015167 = fieldWeight in 3882, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.3414435 = idf(docFreq=566, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.062533475 = weight(abstract_txt:subjects in 3882) [ClassicSimilarity], result of:
           0.062533475 = score(doc=3882,freq=1.0), product of:
             0.14762004 = queryWeight, product of:
               1.1474975 = boost
               5.422221 = idf(docFreq=522, maxDocs=43556)
               0.023725552 = queryNorm
             0.42361104 = fieldWeight in 3882, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.422221 = idf(docFreq=522, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.029342204 = weight(abstract_txt:were in 3882) [ClassicSimilarity], result of:
           0.029342204 = score(doc=3882,freq=1.0), product of:
             0.10203814 = queryWeight, product of:
               1.1684389 = boost
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.023725552 = queryNorm
             0.28756115 = fieldWeight in 3882, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.16620384 = weight(abstract_txt:trec in 3882) [ClassicSimilarity], result of:
           0.16620384 = score(doc=3882,freq=2.0), product of:
             0.22481276 = queryWeight, product of:
               1.4160858 = boost
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.023725552 = queryNorm
             0.73929894 = fieldWeight in 3882, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.10223983 = weight(abstract_txt:relevant in 3882) [ClassicSimilarity], result of:
           0.10223983 = score(doc=3882,freq=3.0), product of:
             0.16260742 = queryWeight, product of:
               1.4750098 = boost
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.023725552 = queryNorm
             0.6287526 = fieldWeight in 3882, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.039122935 = weight(abstract_txt:user in 3882) [ClassicSimilarity], result of:
           0.039122935 = score(doc=3882,freq=1.0), product of:
             0.13605084 = queryWeight, product of:
               1.5579184 = boost
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.023725552 = queryNorm
             0.28756115 = fieldWeight in 3882, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.17514636 = weight(abstract_txt:feedback in 3882) [ClassicSimilarity], result of:
           0.17514636 = score(doc=3882,freq=2.0), product of:
             0.26649645 = queryWeight, product of:
               1.8882977 = boost
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.023725552 = queryNorm
             0.65721834 = fieldWeight in 3882, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.30989903 = weight(abstract_txt:expansion in 3882) [ClassicSimilarity], result of:
           0.30989903 = score(doc=3882,freq=3.0), product of:
             0.37484637 = queryWeight, product of:
               2.5859537 = boost
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.023725552 = queryNorm
             0.8267361 = fieldWeight in 3882, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
         0.23561631 = weight(abstract_txt:relevance in 3882) [ClassicSimilarity], result of:
           0.23561631 = score(doc=3882,freq=4.0), product of:
             0.30560973 = queryWeight, product of:
               2.610554 = boost
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.023725552 = queryNorm
             0.77097124 = fieldWeight in 3882, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.078125 = fieldNorm(doc=3882)
       0.48 = coord(12/25)
   

2. Efthimiadis, E.N.: Interactive query expansion : a user-based evaluation in a relevance feedback environment (2000) 0.52

   0.5218808 = sum of:
     0.5218808 = product of:
       1.0872517 = sum of:
         0.120764375 = weight(abstract_txt:query in 699) [ClassicSimilarity], result of:
           0.120764375 = score(doc=699,freq=17.0), product of:
             0.11293135 = queryWeight, product of:
               1.0036594 = boost
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.023725552 = queryNorm
             1.069361 = fieldWeight in 699, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.051843118 = weight(abstract_txt:effectiveness in 699) [ClassicSimilarity], result of:
           0.051843118 = score(doc=699,freq=2.0), product of:
             0.13115625 = queryWeight, product of:
               1.0816172 = boost
               5.11092 = idf(docFreq=713, maxDocs=43556)
               0.023725552 = queryNorm
             0.39527753 = fieldWeight in 699, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.11092 = idf(docFreq=713, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.016335428 = weight(abstract_txt:more in 699) [ClassicSimilarity], result of:
           0.016335428 = score(doc=699,freq=1.0), product of:
             0.08759023 = queryWeight, product of:
               1.0825615 = boost
               3.4102545 = idf(docFreq=3910, maxDocs=43556)
               0.023725552 = queryNorm
             0.18649828 = fieldWeight in 699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4102545 = idf(docFreq=3910, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.017226009 = weight(abstract_txt:using in 699) [ClassicSimilarity], result of:
           0.017226009 = score(doc=699,freq=1.0), product of:
             0.0907455 = queryWeight, product of:
               1.1018876 = boost
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.023725552 = queryNorm
             0.18982768 = fieldWeight in 699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.04592781 = weight(abstract_txt:were in 699) [ClassicSimilarity], result of:
           0.04592781 = score(doc=699,freq=5.0), product of:
             0.10203814 = queryWeight, product of:
               1.1684389 = boost
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.023725552 = queryNorm
             0.4501044 = fieldWeight in 699, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.035940062 = weight(abstract_txt:three in 699) [ClassicSimilarity], result of:
           0.035940062 = score(doc=699,freq=1.0), product of:
             0.14816804 = queryWeight, product of:
               1.4079978 = boost
               4.435435 = idf(docFreq=1402, maxDocs=43556)
               0.023725552 = queryNorm
             0.24256285 = fieldWeight in 699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.435435 = idf(docFreq=1402, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.020948008 = weight(abstract_txt:system in 699) [ClassicSimilarity], result of:
           0.020948008 = score(doc=699,freq=1.0), product of:
             0.11379156 = queryWeight, product of:
               1.4247843 = boost
               3.3662362 = idf(docFreq=4086, maxDocs=43556)
               0.023725552 = queryNorm
             0.18409105 = fieldWeight in 699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3662362 = idf(docFreq=4086, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.058434933 = weight(abstract_txt:relevant in 699) [ClassicSimilarity], result of:
           0.058434933 = score(doc=699,freq=2.0), product of:
             0.16260742 = queryWeight, product of:
               1.4750098 = boost
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.023725552 = queryNorm
             0.35936204 = fieldWeight in 699, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.038729727 = weight(abstract_txt:user in 699) [ClassicSimilarity], result of:
           0.038729727 = score(doc=699,freq=2.0), product of:
             0.13605084 = queryWeight, product of:
               1.5579184 = boost
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.023725552 = queryNorm
             0.284671 = fieldWeight in 699, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.08669302 = weight(abstract_txt:feedback in 699) [ClassicSimilarity], result of:
           0.08669302 = score(doc=699,freq=1.0), product of:
             0.26649645 = queryWeight, product of:
               1.8882977 = boost
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.023725552 = queryNorm
             0.32530648 = fieldWeight in 699, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.4515744 = weight(abstract_txt:expansion in 699) [ClassicSimilarity], result of:
           0.4515744 = score(doc=699,freq=13.0), product of:
             0.37484637 = queryWeight, product of:
               2.5859537 = boost
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.023725552 = queryNorm
             1.2046919 = fieldWeight in 699, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
         0.1428348 = weight(abstract_txt:relevance in 699) [ClassicSimilarity], result of:
           0.1428348 = score(doc=699,freq=3.0), product of:
             0.30560973 = queryWeight, product of:
               2.610554 = boost
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.023725552 = queryNorm
             0.46737647 = fieldWeight in 699, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.0546875 = fieldNorm(doc=699)
       0.48 = coord(12/25)
   

3. Robertson, S.E.: OKAPI at TREC-3 (1995) 0.50

   0.49517164 = sum of:
     0.49517164 = product of:
       1.2379291 = sum of:
         0.050210856 = weight(abstract_txt:query in 692) [ClassicSimilarity], result of:
           0.050210856 = score(doc=692,freq=1.0), product of:
             0.11293135 = queryWeight, product of:
               1.0036594 = boost
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.023725552 = queryNorm
             0.44461396 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.0295303 = weight(abstract_txt:using in 692) [ClassicSimilarity], result of:
           0.0295303 = score(doc=692,freq=1.0), product of:
             0.0907455 = queryWeight, product of:
               1.1018876 = boost
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.023725552 = queryNorm
             0.3254189 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.07173615 = weight(abstract_txt:experiments in 692) [ClassicSimilarity], result of:
           0.07173615 = score(doc=692,freq=1.0), product of:
             0.14325446 = queryWeight, product of:
               1.1304027 = boost
               5.3414435 = idf(docFreq=566, maxDocs=43556)
               0.023725552 = queryNorm
             0.5007603 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3414435 = idf(docFreq=566, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.24426875 = weight(abstract_txt:trec in 692) [ClassicSimilarity], result of:
           0.24426875 = score(doc=692,freq=3.0), product of:
             0.22481276 = queryWeight, product of:
               1.4160858 = boost
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.023725552 = queryNorm
             1.0865431 = fieldWeight in 692, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.03591087 = weight(abstract_txt:system in 692) [ClassicSimilarity], result of:
           0.03591087 = score(doc=692,freq=1.0), product of:
             0.11379156 = queryWeight, product of:
               1.4247843 = boost
               3.3662362 = idf(docFreq=4086, maxDocs=43556)
               0.023725552 = queryNorm
             0.31558466 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3662362 = idf(docFreq=4086, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.046947524 = weight(abstract_txt:user in 692) [ClassicSimilarity], result of:
           0.046947524 = score(doc=692,freq=1.0), product of:
             0.13605084 = queryWeight, product of:
               1.5579184 = boost
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.023725552 = queryNorm
             0.3450734 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6807828 = idf(docFreq=2983, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.14861661 = weight(abstract_txt:feedback in 692) [ClassicSimilarity], result of:
           0.14861661 = score(doc=692,freq=1.0), product of:
             0.26649645 = queryWeight, product of:
               1.8882977 = boost
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.023725552 = queryNorm
             0.55766827 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.16570048 = weight(abstract_txt:ranking in 692) [ClassicSimilarity], result of:
           0.16570048 = score(doc=692,freq=1.0), product of:
             0.31538567 = queryWeight, product of:
               2.3720021 = boost
               5.6041603 = idf(docFreq=435, maxDocs=43556)
               0.023725552 = queryNorm
             0.52539 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6041603 = idf(docFreq=435, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.30363777 = weight(abstract_txt:expansion in 692) [ClassicSimilarity], result of:
           0.30363777 = score(doc=692,freq=2.0), product of:
             0.37484637 = queryWeight, product of:
               2.5859537 = boost
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.023725552 = queryNorm
             0.8100326 = fieldWeight in 692, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
         0.14136979 = weight(abstract_txt:relevance in 692) [ClassicSimilarity], result of:
           0.14136979 = score(doc=692,freq=1.0), product of:
             0.30560973 = queryWeight, product of:
               2.610554 = boost
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.023725552 = queryNorm
             0.46258277 = fieldWeight in 692, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.09375 = fieldNorm(doc=692)
       0.4 = coord(10/25)
   

4. Xu, B.; Lin, H.; Lin, Y.: Assessment of learning to rank methods for query expansion (2016) 0.44

   0.44053268 = sum of:
     0.44053268 = product of:
       1.1013317 = sum of:
         0.047339246 = weight(abstract_txt:query in 4927) [ClassicSimilarity], result of:
           0.047339246 = score(doc=4927,freq=2.0), product of:
             0.11293135 = queryWeight, product of:
               1.0036594 = boost
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.023725552 = queryNorm
             0.41918606 = fieldWeight in 4927, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.019686867 = weight(abstract_txt:using in 4927) [ClassicSimilarity], result of:
           0.019686867 = score(doc=4927,freq=1.0), product of:
             0.0907455 = queryWeight, product of:
               1.1018876 = boost
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.023725552 = queryNorm
             0.21694592 = fieldWeight in 4927, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4711347 = idf(docFreq=3679, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.041074354 = weight(abstract_txt:three in 4927) [ClassicSimilarity], result of:
           0.041074354 = score(doc=4927,freq=1.0), product of:
             0.14816804 = queryWeight, product of:
               1.4079978 = boost
               4.435435 = idf(docFreq=1402, maxDocs=43556)
               0.023725552 = queryNorm
             0.27721468 = fieldWeight in 4927, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.435435 = idf(docFreq=1402, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.094019085 = weight(abstract_txt:trec in 4927) [ClassicSimilarity], result of:
           0.094019085 = score(doc=4927,freq=1.0), product of:
             0.22481276 = queryWeight, product of:
               1.4160858 = boost
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.023725552 = queryNorm
             0.41821063 = fieldWeight in 4927, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.047222555 = weight(abstract_txt:relevant in 4927) [ClassicSimilarity], result of:
           0.047222555 = score(doc=4927,freq=1.0), product of:
             0.16260742 = queryWeight, product of:
               1.4750098 = boost
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.023725552 = queryNorm
             0.29040837 = fieldWeight in 4927, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.07464583 = weight(abstract_txt:experimental in 4927) [ClassicSimilarity], result of:
           0.07464583 = score(doc=4927,freq=1.0), product of:
             0.22065362 = queryWeight, product of:
               1.7182258 = boost
               5.4127064 = idf(docFreq=527, maxDocs=43556)
               0.023725552 = queryNorm
             0.33829415 = fieldWeight in 4927, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4127064 = idf(docFreq=527, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.17160769 = weight(abstract_txt:feedback in 4927) [ClassicSimilarity], result of:
           0.17160769 = score(doc=4927,freq=3.0), product of:
             0.26649645 = queryWeight, product of:
               1.8882977 = boost
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.023725552 = queryNorm
             0.64393985 = fieldWeight in 4927, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.15622391 = weight(abstract_txt:ranking in 4927) [ClassicSimilarity], result of:
           0.15622391 = score(doc=4927,freq=2.0), product of:
             0.31538567 = queryWeight, product of:
               2.3720021 = boost
               5.6041603 = idf(docFreq=435, maxDocs=43556)
               0.023725552 = queryNorm
             0.49534246 = fieldWeight in 4927, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.6041603 = idf(docFreq=435, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.28627247 = weight(abstract_txt:expansion in 4927) [ClassicSimilarity], result of:
           0.28627247 = score(doc=4927,freq=4.0), product of:
             0.37484637 = queryWeight, product of:
               2.5859537 = boost
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.023725552 = queryNorm
             0.7637061 = fieldWeight in 4927, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
         0.16323976 = weight(abstract_txt:relevance in 4927) [ClassicSimilarity], result of:
           0.16323976 = score(doc=4927,freq=3.0), product of:
             0.30560973 = queryWeight, product of:
               2.610554 = boost
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.023725552 = queryNorm
             0.5341445 = fieldWeight in 4927, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.0625 = fieldNorm(doc=4927)
       0.4 = coord(10/25)
   

5. Robertson, S.E.; Walker, S.; Hancock-Beaulieu, M.M.: Large test collection experiments of an operational, interactive system : OKAPI at TREC (1995) 0.40

   0.39662817 = sum of:
     0.39662817 = product of:
       1.1017449 = sum of:
         0.07100887 = weight(abstract_txt:query in 31) [ClassicSimilarity], result of:
           0.07100887 = score(doc=31,freq=2.0), product of:
             0.11293135 = queryWeight, product of:
               1.0036594 = boost
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.023725552 = queryNorm
             0.62877905 = fieldWeight in 31, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.742549 = idf(docFreq=1031, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.028003592 = weight(abstract_txt:more in 31) [ClassicSimilarity], result of:
           0.028003592 = score(doc=31,freq=1.0), product of:
             0.08759023 = queryWeight, product of:
               1.0825615 = boost
               3.4102545 = idf(docFreq=3910, maxDocs=43556)
               0.023725552 = queryNorm
             0.31971136 = fieldWeight in 31, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4102545 = idf(docFreq=3910, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.07173615 = weight(abstract_txt:experiments in 31) [ClassicSimilarity], result of:
           0.07173615 = score(doc=31,freq=1.0), product of:
             0.14325446 = queryWeight, product of:
               1.1304027 = boost
               5.3414435 = idf(docFreq=566, maxDocs=43556)
               0.023725552 = queryNorm
             0.5007603 = fieldWeight in 31, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3414435 = idf(docFreq=566, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.19944459 = weight(abstract_txt:trec in 31) [ClassicSimilarity], result of:
           0.19944459 = score(doc=31,freq=2.0), product of:
             0.22481276 = queryWeight, product of:
               1.4160858 = boost
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.023725552 = queryNorm
             0.8871587 = fieldWeight in 31, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.69137 = idf(docFreq=146, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.03591087 = weight(abstract_txt:system in 31) [ClassicSimilarity], result of:
           0.03591087 = score(doc=31,freq=1.0), product of:
             0.11379156 = queryWeight, product of:
               1.4247843 = boost
               3.3662362 = idf(docFreq=4086, maxDocs=43556)
               0.023725552 = queryNorm
             0.31558466 = fieldWeight in 31, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3662362 = idf(docFreq=4086, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.07083383 = weight(abstract_txt:relevant in 31) [ClassicSimilarity], result of:
           0.07083383 = score(doc=31,freq=1.0), product of:
             0.16260742 = queryWeight, product of:
               1.4750098 = boost
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.023725552 = queryNorm
             0.43561256 = fieldWeight in 31, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.646534 = idf(docFreq=1135, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.21017562 = weight(abstract_txt:feedback in 31) [ClassicSimilarity], result of:
           0.21017562 = score(doc=31,freq=2.0), product of:
             0.26649645 = queryWeight, product of:
               1.8882977 = boost
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.023725552 = queryNorm
             0.78866196 = fieldWeight in 31, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.9484615 = idf(docFreq=308, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.21470433 = weight(abstract_txt:expansion in 31) [ClassicSimilarity], result of:
           0.21470433 = score(doc=31,freq=1.0), product of:
             0.37484637 = queryWeight, product of:
               2.5859537 = boost
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.023725552 = queryNorm
             0.57277954 = fieldWeight in 31, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1096487 = idf(docFreq=262, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
         0.19992706 = weight(abstract_txt:relevance in 31) [ClassicSimilarity], result of:
           0.19992706 = score(doc=31,freq=2.0), product of:
             0.30560973 = queryWeight, product of:
               2.610554 = boost
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.023725552 = queryNorm
             0.6541908 = fieldWeight in 31, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.934216 = idf(docFreq=851, maxDocs=43556)
               0.09375 = fieldNorm(doc=31)
       0.36 = coord(9/25)