-
Witten, I.H.; Moffat, A.; Bell, T.C.: Managing gigabytes : compressing and indexing documents and images (1994)
0.01
0.0134057235 = product of:
0.04021717 = sum of:
0.04021717 = product of:
0.08043434 = sum of:
0.08043434 = weight(_text_:indexing in 3083) [ClassicSimilarity], result of:
0.08043434 = score(doc=3083,freq=2.0), product of:
0.19018644 = queryWeight, product of:
3.8278677 = idf(docFreq=2614, maxDocs=44218)
0.049684696 = queryNorm
0.42292362 = fieldWeight in 3083, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.8278677 = idf(docFreq=2614, maxDocs=44218)
0.078125 = fieldNorm(doc=3083)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
-
Moffat, A.; Zobel, J.: Self-indexing inverted files for fast text retrieval (1996)
0.01
0.011375135 = product of:
0.034125403 = sum of:
0.034125403 = product of:
0.068250805 = sum of:
0.068250805 = weight(_text_:indexing in 9) [ClassicSimilarity], result of:
0.068250805 = score(doc=9,freq=4.0), product of:
0.19018644 = queryWeight, product of:
3.8278677 = idf(docFreq=2614, maxDocs=44218)
0.049684696 = queryNorm
0.3588626 = fieldWeight in 9, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.8278677 = idf(docFreq=2614, maxDocs=44218)
0.046875 = fieldNorm(doc=9)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Abstract
- Query processing costs on large text databases are dominated by the need to retrieve and scan the inverted list of each query term. Retrieval time for inverted lists can be greatly reduced by the use of compression, but this adds to the CPU time required. Shows that the CPU component of query response time for conjunctive Boolean queries and for informal ranked queries can be similarly reduced, at little cost in terms of storage, by the inclusion of an internal index in each compressed inverted list. This method has been applied in a retrieval system for a collection of nearly 2 million short documents. The self-indexing strategy adds less than 20% to the size of the compressed inverted file, which itself occupies less than 10% of the indexed text, yet can reduce processing time for Boolean queries of 5-10 terms to under one fifth of the previous cost. Similarly, ranked queries of 40-50 terms can be evaluated in as little as 25% of the previous time, with little or no loss of retrieval effectiveness
-
Moffat, A.; Bell, T.A.H.: In situ generation of compressed inverted files (1995)
0.01
0.0080434345 = product of:
0.024130303 = sum of:
0.024130303 = product of:
0.048260607 = sum of:
0.048260607 = weight(_text_:indexing in 2648) [ClassicSimilarity], result of:
0.048260607 = score(doc=2648,freq=2.0), product of:
0.19018644 = queryWeight, product of:
3.8278677 = idf(docFreq=2614, maxDocs=44218)
0.049684696 = queryNorm
0.2537542 = fieldWeight in 2648, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.8278677 = idf(docFreq=2614, maxDocs=44218)
0.046875 = fieldNorm(doc=2648)
0.5 = coord(1/2)
0.33333334 = coord(1/3)
- Abstract
- An inverted index stores, for each term that appears in a collection of documents, a list of document numbers containing that term. Such an index is indispensible when Boolean or informal ranked queries are to be answered. Construction of the index ist, however, a non trivial task. Simple methods using in.memory data structures cannot be used for large collections because they require too much random access storage, and traditional disc based methods require large amounts of temporary file space. Describes a new indexing algorithm designed to create large compressed inverted indexes in situ. It makes use of simple compression codes for the positive integers and an in place external multi way merge sort. The new techniques has been used to invert a 2-gigabyte text collection in under 4 hours, using less than 40 megabytes of temporary disc space, and less than 20 megabytes of main memory
