Document (#39866)

Author

Landvad Clemmensen, M.
Borlund, P.

Title

Order effect in interactive information retrieval evaluation : an empirical study

Source

Journal of documentation. 72(2016) no.2, S.194-213

Year

2016

Abstract

Purpose - This paper reports a study of order effect in interactive information retrieval (IIR) studies. The phenomenon of order effect is well-known, and it is the main reason why searches are permuted (counter-balanced) between test participants in IIR studies. However, the phenomenon is not yet fully understood or investigated in relation to IIR; hence the objective is to increase our knowledge of this phenomenon in the context of IIR as it has implications for test design of IIR studies Design/methodology/approach - Order effect is studied via partly a literature review and partly an empirical IIR study. The empirical IIR study is designed as a classic between-groups design. The IIR search behaviour was logged and complementary post-search interviews were conducted. Findings - The order effect between groups and within search tasks were measured against nine classic IIR performance parameters of search interaction behaviour. Order effect is seen with respect to three performance parameters (website changes, visit of webpages, and formulation of queries) shown by an increase in activity on the last performed search. Further the theories with respect to motivation, fatigue, and the good-subject effect shed light on how and why order effect may affect test participants' IR system interaction and search behaviour. Research implications/limitations - Insight about order effect has implications for test design of IIR studies and hence the knowledge base generated on the basis of such studies. Due to the limited sample of 20 test participants (Library and Information Science (LIS) students) inference statistics is not applicable; hence conclusions can be drawn from this sample of test participants only. Originality/Value - Only few studies in LIS focus on order effect and none from the perspective of IIR.

Similar documents (author)

1. Borlund, P.: ¬The IIR evaluation model : a framework for evaluation of interactive information retrieval systems (2003) 5.58

   5.5776863 = sum of:
     5.5776863 = weight(author_txt:borlund in 922) [ClassicSimilarity], result of:
       5.5776863 = fieldWeight in 922, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.924298 = idf(docFreq=15, maxDocs=44218)
         0.625 = fieldNorm(doc=922)
   

2. Borlund, P.: ¬The concept of relevance in IR (2003) 5.58

   5.5776863 = sum of:
     5.5776863 = weight(author_txt:borlund in 1798) [ClassicSimilarity], result of:
       5.5776863 = fieldWeight in 1798, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.924298 = idf(docFreq=15, maxDocs=44218)
         0.625 = fieldNorm(doc=1798)
   

3. Borlund, P.: Evaluation of interactive information retrieval systems (2000) 5.58

   5.5776863 = sum of:
     5.5776863 = weight(author_txt:borlund in 2556) [ClassicSimilarity], result of:
       5.5776863 = fieldWeight in 2556, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.924298 = idf(docFreq=15, maxDocs=44218)
         0.625 = fieldNorm(doc=2556)
   

4. Borlund, P.: Experimental components for the evaluation of interactive information retrieval systems (2000) 5.58

   5.5776863 = sum of:
     5.5776863 = weight(author_txt:borlund in 4549) [ClassicSimilarity], result of:
       5.5776863 = fieldWeight in 4549, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.924298 = idf(docFreq=15, maxDocs=44218)
         0.625 = fieldNorm(doc=4549)
   

5. Borlund, P.: ¬A study of the use of simulated work task situations in interactive information retrieval evaluations : a meta-evaluation (2016) 5.58

   5.5776863 = sum of:
     5.5776863 = weight(author_txt:borlund in 2880) [ClassicSimilarity], result of:
       5.5776863 = fieldWeight in 2880, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.924298 = idf(docFreq=15, maxDocs=44218)
         0.625 = fieldNorm(doc=2880)
   

Similar documents (content)

1. Borlund, P.: ¬A study of the use of simulated work task situations in interactive information retrieval evaluations : a meta-evaluation (2016) 0.57

   0.5693405 = sum of:
     0.5693405 = product of:
       1.0166795 = sum of:
         0.03443115 = weight(abstract_txt:sample in 2880) [ClassicSimilarity], result of:
           0.03443115 = score(doc=2880,freq=1.0), product of:
             0.11342971 = queryWeight, product of:
               1.2050737 = boost
               5.550558 = idf(docFreq=466, maxDocs=44218)
               0.016958075 = queryNorm
             0.30354613 = fieldWeight in 2880, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.550558 = idf(docFreq=466, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.03451108 = weight(abstract_txt:interactive in 2880) [ClassicSimilarity], result of:
           0.03451108 = score(doc=2880,freq=1.0), product of:
             0.11360518 = queryWeight, product of:
               1.2060055 = boost
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.016958075 = queryNorm
             0.30378085 = fieldWeight in 2880, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.04370649 = weight(abstract_txt:respect in 2880) [ClassicSimilarity], result of:
           0.04370649 = score(doc=2880,freq=1.0), product of:
             0.13298105 = queryWeight, product of:
               1.3048034 = boost
               6.009912 = idf(docFreq=294, maxDocs=44218)
               0.016958075 = queryNorm
             0.32866707 = fieldWeight in 2880, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.009912 = idf(docFreq=294, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.03844685 = weight(abstract_txt:implications in 2880) [ClassicSimilarity], result of:
           0.03844685 = score(doc=2880,freq=2.0), product of:
             0.110922605 = queryWeight, product of:
               1.4595059 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.016958075 = queryNorm
             0.3466097 = fieldWeight in 2880, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.036139656 = weight(abstract_txt:study in 2880) [ClassicSimilarity], result of:
           0.036139656 = score(doc=2880,freq=5.0), product of:
             0.08631808 = queryWeight, product of:
               1.4866753 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.016958075 = queryNorm
             0.41868 = fieldWeight in 2880, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.12047479 = weight(abstract_txt:partly in 2880) [ClassicSimilarity], result of:
           0.12047479 = score(doc=2880,freq=2.0), product of:
             0.20749769 = queryWeight, product of:
               1.6298846 = boost
               7.5072327 = idf(docFreq=65, maxDocs=44218)
               0.016958075 = queryNorm
             0.5806079 = fieldWeight in 2880, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.5072327 = idf(docFreq=65, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.04272948 = weight(abstract_txt:empirical in 2880) [ClassicSimilarity], result of:
           0.04272948 = score(doc=2880,freq=1.0), product of:
             0.14994821 = queryWeight, product of:
               1.6969401 = boost
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.016958075 = queryNorm
             0.28496158 = fieldWeight in 2880, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.048541844 = weight(abstract_txt:design in 2880) [ClassicSimilarity], result of:
           0.048541844 = score(doc=2880,freq=4.0), product of:
             0.113194846 = queryWeight, product of:
               1.7024664 = boost
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.016958075 = queryNorm
             0.4288344 = fieldWeight in 2880, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.08373808 = weight(abstract_txt:hence in 2880) [ClassicSimilarity], result of:
           0.08373808 = score(doc=2880,freq=1.0), product of:
             0.23482169 = queryWeight, product of:
               2.123562 = boost
               6.5207376 = idf(docFreq=176, maxDocs=44218)
               0.016958075 = queryNorm
             0.35660285 = fieldWeight in 2880, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.5207376 = idf(docFreq=176, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.10762251 = weight(abstract_txt:participants in 2880) [ClassicSimilarity], result of:
           0.10762251 = score(doc=2880,freq=3.0), product of:
             0.21183488 = queryWeight, product of:
               2.3289702 = boost
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.016958075 = queryNorm
             0.5080491 = fieldWeight in 2880, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.029567366 = weight(abstract_txt:search in 2880) [ClassicSimilarity], result of:
           0.029567366 = score(doc=2880,freq=1.0), product of:
             0.14780007 = queryWeight, product of:
               2.3825836 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.016958075 = queryNorm
             0.20004974 = fieldWeight in 2880, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.093919106 = weight(abstract_txt:studies in 2880) [ClassicSimilarity], result of:
           0.093919106 = score(doc=2880,freq=4.0), product of:
             0.20119423 = queryWeight, product of:
               2.7798324 = boost
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.016958075 = queryNorm
             0.46680814 = fieldWeight in 2880, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.19016787 = weight(abstract_txt:test in 2880) [ClassicSimilarity], result of:
           0.19016787 = score(doc=2880,freq=6.0), product of:
             0.2813027 = queryWeight, product of:
               3.2869856 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.016958075 = queryNorm
             0.6760258 = fieldWeight in 2880, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
         0.11268324 = weight(abstract_txt:order in 2880) [ClassicSimilarity], result of:
           0.11268324 = score(doc=2880,freq=2.0), product of:
             0.32763693 = queryWeight, product of:
               4.3446307 = boost
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.016958075 = queryNorm
             0.34392715 = fieldWeight in 2880, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.0546875 = fieldNorm(doc=2880)
       0.56 = coord(14/25)
   

2. MacFarlane, A.; Al-Wabil, A.; Marshall, C.R.; Albrair, A.; Jones, S.A.; Zaphiris, P.: ¬The effect of dyslexia on information retrieval : a pilot study (2010) 0.27

   0.2665314 = sum of:
     0.2665314 = product of:
       0.74036497 = sum of:
         0.052861404 = weight(abstract_txt:groups in 3624) [ClassicSimilarity], result of:
           0.052861404 = score(doc=3624,freq=3.0), product of:
             0.09575245 = queryWeight, product of:
               1.1071976 = boost
               5.0997415 = idf(docFreq=732, maxDocs=44218)
               0.016958075 = queryNorm
             0.5520632 = fieldWeight in 3624, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.0997415 = idf(docFreq=732, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.02867852 = weight(abstract_txt:between in 3624) [ClassicSimilarity], result of:
           0.02867852 = score(doc=3624,freq=4.0), product of:
             0.066243924 = queryWeight, product of:
               1.1278955 = boost
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.016958075 = queryNorm
             0.43292302 = fieldWeight in 3624, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.018471023 = weight(abstract_txt:study in 3624) [ClassicSimilarity], result of:
           0.018471023 = score(doc=3624,freq=1.0), product of:
             0.08631808 = queryWeight, product of:
               1.4866753 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.016958075 = queryNorm
             0.21398787 = fieldWeight in 3624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.03922773 = weight(abstract_txt:design in 3624) [ClassicSimilarity], result of:
           0.03922773 = score(doc=3624,freq=2.0), product of:
             0.113194846 = queryWeight, product of:
               1.7024664 = boost
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.016958075 = queryNorm
             0.34655052 = fieldWeight in 3624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.13080287 = weight(abstract_txt:behaviour in 3624) [ClassicSimilarity], result of:
           0.13080287 = score(doc=3624,freq=4.0), product of:
             0.18218799 = queryWeight, product of:
               1.8704914 = boost
               5.7436438 = idf(docFreq=384, maxDocs=44218)
               0.016958075 = queryNorm
             0.71795547 = fieldWeight in 3624, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.7436438 = idf(docFreq=384, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.071012445 = weight(abstract_txt:participants in 3624) [ClassicSimilarity], result of:
           0.071012445 = score(doc=3624,freq=1.0), product of:
             0.21183488 = queryWeight, product of:
               2.3289702 = boost
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.016958075 = queryNorm
             0.33522546 = fieldWeight in 3624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.04778808 = weight(abstract_txt:search in 3624) [ClassicSimilarity], result of:
           0.04778808 = score(doc=3624,freq=2.0), product of:
             0.14780007 = queryWeight, product of:
               2.3825836 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.016958075 = queryNorm
             0.3233292 = fieldWeight in 3624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.09106181 = weight(abstract_txt:order in 3624) [ClassicSimilarity], result of:
           0.09106181 = score(doc=3624,freq=1.0), product of:
             0.32763693 = queryWeight, product of:
               4.3446307 = boost
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.016958075 = queryNorm
             0.27793512 = fieldWeight in 3624, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
         0.2604611 = weight(abstract_txt:effect in 3624) [ClassicSimilarity], result of:
           0.2604611 = score(doc=3624,freq=2.0), product of:
             0.54271656 = queryWeight, product of:
               5.894158 = boost
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.016958075 = queryNorm
             0.479921 = fieldWeight in 3624, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.0625 = fieldNorm(doc=3624)
       0.36 = coord(9/25)
   

3. Xu, Y.; Wang, D.: Order effect in relevance judgment : mediation and causality (2008) 0.26

   0.26100275 = sum of:
     0.26100275 = product of:
       1.0875115 = sum of:
         0.04930154 = weight(abstract_txt:interactive in 1877) [ClassicSimilarity], result of:
           0.04930154 = score(doc=1877,freq=1.0), product of:
             0.11360518 = queryWeight, product of:
               1.2060055 = boost
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.016958075 = queryNorm
             0.43397263 = fieldWeight in 1877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5548496 = idf(docFreq=464, maxDocs=44218)
               0.078125 = fieldNorm(doc=1877)
         0.038837187 = weight(abstract_txt:implications in 1877) [ClassicSimilarity], result of:
           0.038837187 = score(doc=1877,freq=1.0), product of:
             0.110922605 = queryWeight, product of:
               1.4595059 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.016958075 = queryNorm
             0.35012868 = fieldWeight in 1877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.078125 = fieldNorm(doc=1877)
         0.032652464 = weight(abstract_txt:study in 1877) [ClassicSimilarity], result of:
           0.032652464 = score(doc=1877,freq=2.0), product of:
             0.08631808 = queryWeight, product of:
               1.4866753 = boost
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.016958075 = queryNorm
             0.3782807 = fieldWeight in 1877, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.423806 = idf(docFreq=3916, maxDocs=44218)
               0.078125 = fieldNorm(doc=1877)
         0.061042115 = weight(abstract_txt:empirical in 1877) [ClassicSimilarity], result of:
           0.061042115 = score(doc=1877,freq=1.0), product of:
             0.14994821 = queryWeight, product of:
               1.6969401 = boost
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.016958075 = queryNorm
             0.40708798 = fieldWeight in 1877, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.078125 = fieldNorm(doc=1877)
         0.2545255 = weight(abstract_txt:order in 1877) [ClassicSimilarity], result of:
           0.2545255 = score(doc=1877,freq=5.0), product of:
             0.32763693 = queryWeight, product of:
               4.3446307 = boost
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.016958075 = queryNorm
             0.7768523 = fieldWeight in 1877, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.078125 = fieldNorm(doc=1877)
         0.6511527 = weight(abstract_txt:effect in 1877) [ClassicSimilarity], result of:
           0.6511527 = score(doc=1877,freq=8.0), product of:
             0.54271656 = queryWeight, product of:
               5.894158 = boost
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.016958075 = queryNorm
             1.1998025 = fieldWeight in 1877, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.078125 = fieldNorm(doc=1877)
       0.24 = coord(6/25)
   

4. Jansen, B.J.; Zhang, M.; Schultz, C.D.: Brand and its effect on user perception of search engine performance (2009) 0.23

   0.23174888 = sum of:
     0.23174888 = product of:
       0.72421527 = sum of:
         0.03956902 = weight(abstract_txt:performance in 2948) [ClassicSimilarity], result of:
           0.03956902 = score(doc=2948,freq=3.0), product of:
             0.07893948 = queryWeight, product of:
               1.0053039 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.016958075 = queryNorm
             0.50125766 = fieldWeight in 2948, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.031069748 = weight(abstract_txt:implications in 2948) [ClassicSimilarity], result of:
           0.031069748 = score(doc=2948,freq=1.0), product of:
             0.110922605 = queryWeight, product of:
               1.4595059 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.016958075 = queryNorm
             0.28010294 = fieldWeight in 2948, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.04883369 = weight(abstract_txt:empirical in 2948) [ClassicSimilarity], result of:
           0.04883369 = score(doc=2948,freq=1.0), product of:
             0.14994821 = queryWeight, product of:
               1.6969401 = boost
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.016958075 = queryNorm
             0.3256704 = fieldWeight in 2948, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.03922773 = weight(abstract_txt:design in 2948) [ClassicSimilarity], result of:
           0.03922773 = score(doc=2948,freq=2.0), product of:
             0.113194846 = queryWeight, product of:
               1.7024664 = boost
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.016958075 = queryNorm
             0.34655052 = fieldWeight in 2948, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.071012445 = weight(abstract_txt:participants in 2948) [ClassicSimilarity], result of:
           0.071012445 = score(doc=2948,freq=1.0), product of:
             0.21183488 = queryWeight, product of:
               2.3289702 = boost
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.016958075 = queryNorm
             0.33522546 = fieldWeight in 2948, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3636074 = idf(docFreq=562, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.12183618 = weight(abstract_txt:search in 2948) [ClassicSimilarity], result of:
           0.12183618 = score(doc=2948,freq=13.0), product of:
             0.14780007 = queryWeight, product of:
               2.3825836 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.016958075 = queryNorm
             0.824331 = fieldWeight in 2948, product of:
               3.6055512 = tf(freq=13.0), with freq of:
                 13.0 = termFreq=13.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.05366806 = weight(abstract_txt:studies in 2948) [ClassicSimilarity], result of:
           0.05366806 = score(doc=2948,freq=1.0), product of:
             0.20119423 = queryWeight, product of:
               2.7798324 = boost
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.016958075 = queryNorm
             0.2667475 = fieldWeight in 2948, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
         0.31899837 = weight(abstract_txt:effect in 2948) [ClassicSimilarity], result of:
           0.31899837 = score(doc=2948,freq=3.0), product of:
             0.54271656 = queryWeight, product of:
               5.894158 = boost
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.016958075 = queryNorm
             0.5877808 = fieldWeight in 2948, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.0625 = fieldNorm(doc=2948)
       0.32 = coord(8/25)
   

5. Belkin, N.J.: ¬The use of multiple information problem representation for information retrieval (1994) 0.22

   0.22196905 = sum of:
     0.22196905 = product of:
       0.6936533 = sum of:
         0.04038496 = weight(abstract_txt:performance in 8579) [ClassicSimilarity], result of:
           0.04038496 = score(doc=8579,freq=2.0), product of:
             0.07893948 = queryWeight, product of:
               1.0053039 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.016958075 = queryNorm
             0.51159394 = fieldWeight in 8579, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.038837187 = weight(abstract_txt:implications in 8579) [ClassicSimilarity], result of:
           0.038837187 = score(doc=8579,freq=1.0), product of:
             0.110922605 = queryWeight, product of:
               1.4595059 = boost
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.016958075 = queryNorm
             0.35012868 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.481647 = idf(docFreq=1359, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.061042115 = weight(abstract_txt:empirical in 8579) [ClassicSimilarity], result of:
           0.061042115 = score(doc=8579,freq=1.0), product of:
             0.14994821 = queryWeight, product of:
               1.6969401 = boost
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.016958075 = queryNorm
             0.40708798 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2107263 = idf(docFreq=655, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.03467275 = weight(abstract_txt:design in 8579) [ClassicSimilarity], result of:
           0.03467275 = score(doc=8579,freq=1.0), product of:
             0.113194846 = queryWeight, product of:
               1.7024664 = boost
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.016958075 = queryNorm
             0.3063103 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.9207718 = idf(docFreq=2382, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.10758667 = weight(abstract_txt:phenomenon in 8579) [ClassicSimilarity], result of:
           0.10758667 = score(doc=8579,freq=1.0), product of:
             0.21878986 = queryWeight, product of:
               2.0497904 = boost
               6.29421 = idf(docFreq=221, maxDocs=44218)
               0.016958075 = queryNorm
             0.49173516 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.29421 = idf(docFreq=221, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.06708507 = weight(abstract_txt:studies in 8579) [ClassicSimilarity], result of:
           0.06708507 = score(doc=8579,freq=1.0), product of:
             0.20119423 = queryWeight, product of:
               2.7798324 = boost
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.016958075 = queryNorm
             0.33343437 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.26796 = idf(docFreq=1683, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.113827266 = weight(abstract_txt:order in 8579) [ClassicSimilarity], result of:
           0.113827266 = score(doc=8579,freq=1.0), product of:
             0.32763693 = queryWeight, product of:
               4.3446307 = boost
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.016958075 = queryNorm
             0.3474189 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
         0.23021726 = weight(abstract_txt:effect in 8579) [ClassicSimilarity], result of:
           0.23021726 = score(doc=8579,freq=1.0), product of:
             0.54271656 = queryWeight, product of:
               5.894158 = boost
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.016958075 = queryNorm
             0.42419428 = fieldWeight in 8579, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4296865 = idf(docFreq=526, maxDocs=44218)
               0.078125 = fieldNorm(doc=8579)
       0.32 = coord(8/25)