Document (#14178)

Author

MacFarlane, A.
Robertson, S.E.
McCann, J.A.

Title

Parallel computing for passage retrieval

Source

Aslib proceedings. 56(2004) no.4, S.201-211

Year

2004

Abstract

In this paper methods for both speeding up passage processing and examining more passages using parallel computers are explored. The number of passages processed are varied in order to examine the effect on retrieval effectiveness and efficiency. The particular algorithm applied has previously been used to good effect in Okapi experiments at TREC. This algorithm and the mechanism for applying parallel computing to speed up processing are described.

Theme

Retrievalalgorithmen

Object

Okapi
TREC

Similar documents (author)

1. MacFarlane, A.; Robertson, S.E.; McCann, J.A.: Parallel computing in information retrieval : an updated review (1997) 4.25

   4.246812 = sum of:
     4.246812 = sum of:
       1.5681727 = weight(author_txt:robertson in 520) [ClassicSimilarity], result of:
         1.5681727 = score(doc=520,freq=1.0), product of:
           0.5733635 = queryWeight, product of:
             7.2934427 = idf(docFreq=78, maxDocs=42740)
             0.07861356 = queryNorm
           2.7350411 = fieldWeight in 520, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.2934427 = idf(docFreq=78, maxDocs=42740)
             0.375 = fieldNorm(doc=520)
       2.678639 = weight(author_txt:macfarlane in 520) [ClassicSimilarity], result of:
         2.678639 = score(doc=520,freq=1.0), product of:
           0.8193012 = queryWeight, product of:
             1.1953822 = boost
             8.7184515 = idf(docFreq=18, maxDocs=42740)
             0.07861356 = queryNorm
           3.2694192 = fieldWeight in 520, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.7184515 = idf(docFreq=18, maxDocs=42740)
             0.375 = fieldNorm(doc=520)
   

2. MacFarlane, A.; McCann, J.A.; Robertson, S.E.: Parallel methods for the generation of partitioned inverted files (2005) 4.25

   4.246812 = sum of:
     4.246812 = sum of:
       1.5681727 = weight(author_txt:robertson in 1777) [ClassicSimilarity], result of:
         1.5681727 = score(doc=1777,freq=1.0), product of:
           0.5733635 = queryWeight, product of:
             7.2934427 = idf(docFreq=78, maxDocs=42740)
             0.07861356 = queryNorm
           2.7350411 = fieldWeight in 1777, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.2934427 = idf(docFreq=78, maxDocs=42740)
             0.375 = fieldNorm(doc=1777)
       2.678639 = weight(author_txt:macfarlane in 1777) [ClassicSimilarity], result of:
         2.678639 = score(doc=1777,freq=1.0), product of:
           0.8193012 = queryWeight, product of:
             1.1953822 = boost
             8.7184515 = idf(docFreq=18, maxDocs=42740)
             0.07861356 = queryNorm
           3.2694192 = fieldWeight in 1777, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.7184515 = idf(docFreq=18, maxDocs=42740)
             0.375 = fieldNorm(doc=1777)
   

3. MacFarlane, A.; McCann, J.A.; Robertson, S.E.: Parallel methods for the update of partitioned inverted files (2007) 4.25

   4.246812 = sum of:
     4.246812 = sum of:
       1.5681727 = weight(author_txt:robertson in 2820) [ClassicSimilarity], result of:
         1.5681727 = score(doc=2820,freq=1.0), product of:
           0.5733635 = queryWeight, product of:
             7.2934427 = idf(docFreq=78, maxDocs=42740)
             0.07861356 = queryNorm
           2.7350411 = fieldWeight in 2820, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.2934427 = idf(docFreq=78, maxDocs=42740)
             0.375 = fieldNorm(doc=2820)
       2.678639 = weight(author_txt:macfarlane in 2820) [ClassicSimilarity], result of:
         2.678639 = score(doc=2820,freq=1.0), product of:
           0.8193012 = queryWeight, product of:
             1.1953822 = boost
             8.7184515 = idf(docFreq=18, maxDocs=42740)
             0.07861356 = queryNorm
           3.2694192 = fieldWeight in 2820, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.7184515 = idf(docFreq=18, maxDocs=42740)
             0.375 = fieldNorm(doc=2820)
   

4. MacFarlane, A.: On open source IR (2003) 2.23

   2.2321994 = sum of:
     2.2321994 = product of:
       4.464399 = sum of:
         4.464399 = weight(author_txt:macfarlane in 3011) [ClassicSimilarity], result of:
           4.464399 = score(doc=3011,freq=1.0), product of:
             0.8193012 = queryWeight, product of:
               1.1953822 = boost
               8.7184515 = idf(docFreq=18, maxDocs=42740)
               0.07861356 = queryNorm
             5.4490323 = fieldWeight in 3011, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.7184515 = idf(docFreq=18, maxDocs=42740)
               0.625 = fieldNorm(doc=3011)
       0.5 = coord(1/2)
   

5. MacFarlane, A.: Evaluation of web search for the information practitioner (2007) 2.23

   2.2321994 = sum of:
     2.2321994 = product of:
       4.464399 = sum of:
         4.464399 = weight(author_txt:macfarlane in 2818) [ClassicSimilarity], result of:
           4.464399 = score(doc=2818,freq=1.0), product of:
             0.8193012 = queryWeight, product of:
               1.1953822 = boost
               8.7184515 = idf(docFreq=18, maxDocs=42740)
               0.07861356 = queryNorm
             5.4490323 = fieldWeight in 2818, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.7184515 = idf(docFreq=18, maxDocs=42740)
               0.625 = fieldNorm(doc=2818)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Melucci, M.: Passage retrieval : a probabilistic technique (1998) 0.23

   0.22898988 = sum of:
     0.22898988 = product of:
       0.95412457 = sum of:
         0.032481257 = weight(abstract_txt:effectiveness in 2151) [ClassicSimilarity], result of:
           0.032481257 = score(doc=2151,freq=1.0), product of:
             0.08124217 = queryWeight, product of:
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.015875235 = queryNorm
             0.39980787 = fieldWeight in 2151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.078125 = fieldNorm(doc=2151)
         0.03747963 = weight(abstract_txt:experiments in 2151) [ClassicSimilarity], result of:
           0.03747963 = score(doc=2151,freq=1.0), product of:
             0.08937646 = queryWeight, product of:
               1.048868 = boost
               5.3676248 = idf(docFreq=541, maxDocs=42740)
               0.015875235 = queryNorm
             0.41934568 = fieldWeight in 2151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3676248 = idf(docFreq=541, maxDocs=42740)
               0.078125 = fieldNorm(doc=2151)
         0.020161087 = weight(abstract_txt:retrieval in 2151) [ClassicSimilarity], result of:
           0.020161087 = score(doc=2151,freq=1.0), product of:
             0.07448101 = queryWeight, product of:
               1.3540884 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.015875235 = queryNorm
             0.2706876 = fieldWeight in 2151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.078125 = fieldNorm(doc=2151)
         0.09071681 = weight(abstract_txt:algorithm in 2151) [ClassicSimilarity], result of:
           0.09071681 = score(doc=2151,freq=1.0), product of:
             0.20299941 = queryWeight, product of:
               2.2354844 = boost
               5.7200913 = idf(docFreq=380, maxDocs=42740)
               0.015875235 = queryNorm
             0.44688213 = fieldWeight in 2151, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.7200913 = idf(docFreq=380, maxDocs=42740)
               0.078125 = fieldNorm(doc=2151)
         0.38196173 = weight(abstract_txt:passage in 2151) [ClassicSimilarity], result of:
           0.38196173 = score(doc=2151,freq=2.0), product of:
             0.4201195 = queryWeight, product of:
               3.2159603 = boost
               8.228904 = idf(docFreq=30, maxDocs=42740)
               0.015875235 = queryNorm
             0.909174 = fieldWeight in 2151, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.228904 = idf(docFreq=30, maxDocs=42740)
               0.078125 = fieldNorm(doc=2151)
         0.391324 = weight(abstract_txt:passages in 2151) [ClassicSimilarity], result of:
           0.391324 = score(doc=2151,freq=2.0), product of:
             0.4269568 = queryWeight, product of:
               3.242024 = boost
               8.295595 = idf(docFreq=28, maxDocs=42740)
               0.015875235 = queryNorm
             0.9165424 = fieldWeight in 2151, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.295595 = idf(docFreq=28, maxDocs=42740)
               0.078125 = fieldNorm(doc=2151)
       0.24 = coord(6/25)
   

2. Kaszkiel, M.; Zobel, J.: Effective ranking with arbitrary passages (2001) 0.18

   0.17886728 = sum of:
     0.17886728 = product of:
       0.8943364 = sum of:
         0.025985006 = weight(abstract_txt:effectiveness in 680) [ClassicSimilarity], result of:
           0.025985006 = score(doc=680,freq=1.0), product of:
             0.08124217 = queryWeight, product of:
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.015875235 = queryNorm
             0.3198463 = fieldWeight in 680, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.117541 = idf(docFreq=695, maxDocs=42740)
               0.0625 = fieldNorm(doc=680)
         0.029983705 = weight(abstract_txt:experiments in 680) [ClassicSimilarity], result of:
           0.029983705 = score(doc=680,freq=1.0), product of:
             0.08937646 = queryWeight, product of:
               1.048868 = boost
               5.3676248 = idf(docFreq=541, maxDocs=42740)
               0.015875235 = queryNorm
             0.33547655 = fieldWeight in 680, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3676248 = idf(docFreq=541, maxDocs=42740)
               0.0625 = fieldNorm(doc=680)
         0.022809666 = weight(abstract_txt:retrieval in 680) [ClassicSimilarity], result of:
           0.022809666 = score(doc=680,freq=2.0), product of:
             0.07448101 = queryWeight, product of:
               1.3540884 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.015875235 = queryNorm
             0.30624807 = fieldWeight in 680, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.0625 = fieldNorm(doc=680)
         0.43214035 = weight(abstract_txt:passage in 680) [ClassicSimilarity], result of:
           0.43214035 = score(doc=680,freq=4.0), product of:
             0.4201195 = queryWeight, product of:
               3.2159603 = boost
               8.228904 = idf(docFreq=30, maxDocs=42740)
               0.015875235 = queryNorm
             1.028613 = fieldWeight in 680, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               8.228904 = idf(docFreq=30, maxDocs=42740)
               0.0625 = fieldNorm(doc=680)
         0.38341767 = weight(abstract_txt:passages in 680) [ClassicSimilarity], result of:
           0.38341767 = score(doc=680,freq=3.0), product of:
             0.4269568 = queryWeight, product of:
               3.242024 = boost
               8.295595 = idf(docFreq=28, maxDocs=42740)
               0.015875235 = queryNorm
             0.8980245 = fieldWeight in 680, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.295595 = idf(docFreq=28, maxDocs=42740)
               0.0625 = fieldNorm(doc=680)
       0.2 = coord(5/25)
   

3. MacFarlane, A.; McCann, J.A.; Robertson, S.E.: Parallel methods for the generation of partitioned inverted files (2005) 0.18

   0.17583126 = sum of:
     0.17583126 = product of:
       0.6279688 = sum of:
         0.04142601 = weight(abstract_txt:examine in 1777) [ClassicSimilarity], result of:
           0.04142601 = score(doc=1777,freq=2.0), product of:
             0.08799777 = queryWeight, product of:
               1.0407468 = boost
               5.3260646 = idf(docFreq=564, maxDocs=42740)
               0.015875235 = queryNorm
             0.47076204 = fieldWeight in 1777, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.3260646 = idf(docFreq=564, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
         0.029983705 = weight(abstract_txt:experiments in 1777) [ClassicSimilarity], result of:
           0.029983705 = score(doc=1777,freq=1.0), product of:
             0.08937646 = queryWeight, product of:
               1.048868 = boost
               5.3676248 = idf(docFreq=541, maxDocs=42740)
               0.015875235 = queryNorm
             0.33547655 = fieldWeight in 1777, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3676248 = idf(docFreq=541, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
         0.044477403 = weight(abstract_txt:efficiency in 1777) [ClassicSimilarity], result of:
           0.044477403 = score(doc=1777,freq=1.0), product of:
             0.11624989 = queryWeight, product of:
               1.1962048 = boost
               6.121627 = idf(docFreq=254, maxDocs=42740)
               0.015875235 = queryNorm
             0.38260168 = fieldWeight in 1777, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.121627 = idf(docFreq=254, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
         0.07816742 = weight(abstract_txt:speed in 1777) [ClassicSimilarity], result of:
           0.07816742 = score(doc=1777,freq=2.0), product of:
             0.1343714 = queryWeight, product of:
               1.2860643 = boost
               6.581486 = idf(docFreq=160, maxDocs=42740)
               0.015875235 = queryNorm
             0.5817267 = fieldWeight in 1777, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.581486 = idf(docFreq=160, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
         0.022809666 = weight(abstract_txt:retrieval in 1777) [ClassicSimilarity], result of:
           0.022809666 = score(doc=1777,freq=2.0), product of:
             0.07448101 = queryWeight, product of:
               1.3540884 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.015875235 = queryNorm
             0.30624807 = fieldWeight in 1777, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
         0.14634085 = weight(abstract_txt:computing in 1777) [ClassicSimilarity], result of:
           0.14634085 = score(doc=1777,freq=3.0), product of:
             0.22465335 = queryWeight, product of:
               2.3516934 = boost
               6.0174437 = idf(docFreq=282, maxDocs=42740)
               0.015875235 = queryNorm
             0.65140736 = fieldWeight in 1777, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.0174437 = idf(docFreq=282, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
         0.26476374 = weight(abstract_txt:parallel in 1777) [ClassicSimilarity], result of:
           0.26476374 = score(doc=1777,freq=3.0), product of:
             0.38183168 = queryWeight, product of:
               3.7549646 = boost
               6.405395 = idf(docFreq=191, maxDocs=42740)
               0.015875235 = queryNorm
             0.6934043 = fieldWeight in 1777, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.405395 = idf(docFreq=191, maxDocs=42740)
               0.0625 = fieldNorm(doc=1777)
       0.28 = coord(7/25)
   

4. Mengle, S.; Goharian, N.: Passage detection using text classification (2009) 0.18

   0.1756443 = sum of:
     0.1756443 = product of:
       1.4637024 = sum of:
         0.03155709 = weight(abstract_txt:retrieval in 4766) [ClassicSimilarity], result of:
           0.03155709 = score(doc=4766,freq=5.0), product of:
             0.07448101 = queryWeight, product of:
               1.3540884 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.015875235 = queryNorm
             0.42369312 = fieldWeight in 4766, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.0546875 = fieldNorm(doc=4766)
         0.70740306 = weight(abstract_txt:passage in 4766) [ClassicSimilarity], result of:
           0.70740306 = score(doc=4766,freq=14.0), product of:
             0.4201195 = queryWeight, product of:
               3.2159603 = boost
               8.228904 = idf(docFreq=30, maxDocs=42740)
               0.015875235 = queryNorm
             1.6838139 = fieldWeight in 4766, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               8.228904 = idf(docFreq=30, maxDocs=42740)
               0.0546875 = fieldNorm(doc=4766)
         0.7247423 = weight(abstract_txt:passages in 4766) [ClassicSimilarity], result of:
           0.7247423 = score(doc=4766,freq=14.0), product of:
             0.4269568 = queryWeight, product of:
               3.242024 = boost
               8.295595 = idf(docFreq=28, maxDocs=42740)
               0.015875235 = queryNorm
             1.6974604 = fieldWeight in 4766, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               8.295595 = idf(docFreq=28, maxDocs=42740)
               0.0546875 = fieldNorm(doc=4766)
       0.12 = coord(3/25)
   

5. MacFarlane, A.; Robertson, S.E.; McCann, J.A.: Parallel computing in information retrieval : an updated review (1997) 0.17

   0.17160335 = sum of:
     0.17160335 = product of:
       1.072521 = sum of:
         0.055872038 = weight(abstract_txt:retrieval in 520) [ClassicSimilarity], result of:
           0.055872038 = score(doc=520,freq=3.0), product of:
             0.07448101 = queryWeight, product of:
               1.3540884 = boost
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.015875235 = queryNorm
             0.75015146 = fieldWeight in 520, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4648013 = idf(docFreq=3633, maxDocs=42740)
               0.125 = fieldNorm(doc=520)
         0.09405832 = weight(abstract_txt:processing in 520) [ClassicSimilarity], result of:
           0.09405832 = score(doc=520,freq=1.0), product of:
             0.15201546 = queryWeight, product of:
               1.9344974 = boost
               4.9499345 = idf(docFreq=822, maxDocs=42740)
               0.015875235 = queryNorm
             0.6187418 = fieldWeight in 520, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9499345 = idf(docFreq=822, maxDocs=42740)
               0.125 = fieldNorm(doc=520)
         0.23897362 = weight(abstract_txt:computing in 520) [ClassicSimilarity], result of:
           0.23897362 = score(doc=520,freq=2.0), product of:
             0.22465335 = queryWeight, product of:
               2.3516934 = boost
               6.0174437 = idf(docFreq=282, maxDocs=42740)
               0.015875235 = queryNorm
             1.0637438 = fieldWeight in 520, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.0174437 = idf(docFreq=282, maxDocs=42740)
               0.125 = fieldNorm(doc=520)
         0.68361706 = weight(abstract_txt:parallel in 520) [ClassicSimilarity], result of:
           0.68361706 = score(doc=520,freq=5.0), product of:
             0.38183168 = queryWeight, product of:
               3.7549646 = boost
               6.405395 = idf(docFreq=191, maxDocs=42740)
               0.015875235 = queryNorm
             1.7903624 = fieldWeight in 520, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               6.405395 = idf(docFreq=191, maxDocs=42740)
               0.125 = fieldNorm(doc=520)
       0.16 = coord(4/25)