Document (#34464)

Author

Dumais, S.T.

Title

Latent semantic analysis

Source

Annual review of information science and technology. 38(2004), S.189-230

Year

2003

Abstract

Latent Semantic Analysis (LSA) was first introduced in Dumais, Furnas, Landauer, and Deerwester (1988) and Deerwester, Dumais, Furnas, Landauer, and Harshman (1990) as a technique for improving information retrieval. The key insight in LSA was to reduce the dimensionality of the information retrieval problem. Most approaches to retrieving information depend an a lexical match between words in the user's query and those in documents. Indeed, this lexical matching is the way that the popular Web and enterprise search engines work. Such systems are, however, far from ideal. We are all aware of the tremendous amount of irrelevant information that is retrieved when searching. We also fail to find much of the existing relevant material. LSA was designed to address these retrieval problems, using dimension reduction techniques. Fundamental characteristics of human word usage underlie these retrieval failures. People use a wide variety of words to describe the same object or concept (synonymy). Furnas, Landauer, Gomez, and Dumais (1987) showed that people generate the same keyword to describe well-known objects only 20 percent of the time. Poor agreement was also observed in studies of inter-indexer consistency (e.g., Chan, 1989; Tarr & Borko, 1974) in the generation of search terms (e.g., Fidel, 1985; Bates, 1986), and in the generation of hypertext links (Furner, Ellis, & Willett, 1999). Because searchers and authors often use different words, relevant materials are missed. Someone looking for documents an "human-computer interaction" will not find articles that use only the phrase "man-machine studies" or "human factors." People also use the same word to refer to different things (polysemy). Words like "saturn," "jaguar," or "chip" have several different meanings. A short query like "saturn" will thus return many irrelevant documents. The query "Saturn Gar" will return fewer irrelevant items, but it will miss some documents that use only the terms "Saturn automobile." In searching, there is a constant tension between being overly specific and missing relevant information, and being more general and returning irrelevant information.
A number of approaches have been developed in information retrieval to address the problems caused by the variability in word usage. Stemming is a popular technique used to normalize some kinds of surface-level variability by converting words to their morphological root. For example, the words "retrieve," "retrieval," "retrieved," and "retrieving" would all be converted to their root form, "retrieve." The root form is used for both document and query processing. Stemming sometimes helps retrieval, although not much (Harman, 1991; Hull, 1996). And, it does not address Gases where related words are not morphologically related (e.g., physician and doctor). Controlled vocabularies have also been used to limit variability by requiring that query and index terms belong to a pre-defined set of terms. Documents are indexed by a specified or authorized list of subject headings or index terms, called the controlled vocabulary. Library of Congress Subject Headings, Medical Subject Headings, Association for Computing Machinery (ACM) keywords, and Yellow Pages headings are examples of controlled vocabularies. If searchers can find the right controlled vocabulary terms, they do not have to think of all the morphologically related or synonymous terms that authors might have used. However, assigning controlled vocabulary terms in a consistent and thorough manner is a time-consuming and usually manual process. A good deal of research has been published about the effectiveness of controlled vocabulary indexing compared to full text indexing (e.g., Bates, 1998; Lancaster, 1986; Svenonius, 1986). The combination of both full text and controlled vocabularies is often better than either alone, although the size of the advantage is variable (Lancaster, 1986; Markey, Atherton, & Newton, 1982; Srinivasan, 1996). Richer thesauri have also been used to provide synonyms, generalizations, and specializations of users' search terms (see Srinivasan, 1992, for a review). Controlled vocabularies and thesaurus entries can be generated either manually or by the automatic analysis of large collections of texts.
With the advent of large-scale collections of full text, statistical approaches are being used more and more to analyze the relationships among terms and documents. LSA takes this approach. LSA induces knowledge about the meanings of documents and words by analyzing large collections of texts. The approach simultaneously models the relationships among documents based an their constituent words, and the relationships between words based an their occurrence in documents. By using fewer dimensions for representation than there are unique words, LSA induces similarities among terms that are useful in solving the information retrieval problems described earlier. LSA is a fully automatic statistical approach to extracting relations among words by means of their contexts of use in documents, passages, or sentences. It makes no use of natural language processing techniques for analyzing morphological, syntactic, or semantic relations. Nor does it use humanly constructed resources like dictionaries, thesauri, lexical reference systems (e.g., WordNet), semantic networks, or other knowledge representations. Its only input is large amounts of texts. LSA is an unsupervised learning technique. It starts with a large collection of texts, builds a term-document matrix, and tries to uncover some similarity structures that are useful for information retrieval and related text-analysis problems. Several recent ARIST chapters have focused an text mining and discovery (Benoit, 2002; Solomon, 2002; Trybula, 2000). These chapters provide complementary coverage of the field of text analysis.

Theme

Literaturübersicht

Object

Latent Semantic Indexing

Similar documents (author)

1. Gordon, M.D.; Dumais, S.: Using latent semantic indexing for literature based discovery (1998) 4.68

   4.678465 = sum of:
     4.678465 = weight(author_txt:dumais in 5893) [ClassicSimilarity], result of:
       4.678465 = score(doc=5893,freq=1.0), product of:
         0.99999994 = queryWeight, product of:
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.10687265 = queryNorm
         4.6784654 = fieldWeight in 5893, product of:
           1.0 = tf(freq=1.0), with freq of:
             1.0 = termFreq=1.0
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.5 = fieldNorm(doc=5893)
   

2. Dumais, S.; Chen, H.: Hierarchical classification of Web content (2000) 4.68

   4.678465 = sum of:
     4.678465 = weight(author_txt:dumais in 1493) [ClassicSimilarity], result of:
       4.678465 = score(doc=1493,freq=1.0), product of:
         0.99999994 = queryWeight, product of:
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.10687265 = queryNorm
         4.6784654 = fieldWeight in 1493, product of:
           1.0 = tf(freq=1.0), with freq of:
             1.0 = termFreq=1.0
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.5 = fieldNorm(doc=1493)
   

3. Dumais, S.T.; Belkin, N.J.: ¬The TREC interactive tracks : putting the user into search (2005) 4.68

   4.678465 = sum of:
     4.678465 = weight(author_txt:dumais in 82) [ClassicSimilarity], result of:
       4.678465 = score(doc=82,freq=1.0), product of:
         0.99999994 = queryWeight, product of:
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.10687265 = queryNorm
         4.6784654 = fieldWeight in 82, product of:
           1.0 = tf(freq=1.0), with freq of:
             1.0 = termFreq=1.0
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.5 = fieldNorm(doc=82)
   

4. Teevan, J.; Dumais, S.: Web retrieval, ranking and personalization (2011) 4.68

   4.678465 = sum of:
     4.678465 = weight(author_txt:dumais in 1552) [ClassicSimilarity], result of:
       4.678465 = score(doc=1552,freq=1.0), product of:
         0.99999994 = queryWeight, product of:
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.10687265 = queryNorm
         4.6784654 = fieldWeight in 1552, product of:
           1.0 = tf(freq=1.0), with freq of:
             1.0 = termFreq=1.0
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.5 = fieldNorm(doc=1552)
   

5. Berry, M.W.; Dumais, S.T.; O'Brien, G.W.: Using linear algebra for intelligent information retrieval (1995) 3.51

   3.508849 = sum of:
     3.508849 = weight(author_txt:dumais in 3207) [ClassicSimilarity], result of:
       3.508849 = score(doc=3207,freq=1.0), product of:
         0.99999994 = queryWeight, product of:
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.10687265 = queryNorm
         3.5088491 = fieldWeight in 3207, product of:
           1.0 = tf(freq=1.0), with freq of:
             1.0 = termFreq=1.0
           9.356931 = idf(docFreq=9, maxDocs=42596)
           0.375 = fieldNorm(doc=3207)
   

Similar documents (content)

1. Krovetz, R.; Croft, W.B.: Lexical ambiguity and information retrieval (1992) 0.33

   0.33348387 = sum of:
     0.33348387 = product of:
       0.83370966 = sum of:
         0.030980969 = weight(abstract_txt:analysis in 4028) [ClassicSimilarity], result of:
           0.030980969 = score(doc=4028,freq=1.0), product of:
             0.08956951 = queryWeight, product of:
               1.0251547 = boost
               3.6894662 = idf(docFreq=2892, maxDocs=42596)
               0.023681393 = queryNorm
             0.34588745 = fieldWeight in 4028, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6894662 = idf(docFreq=2892, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.14652343 = weight(abstract_txt:lexical in 4028) [ClassicSimilarity], result of:
           0.14652343 = score(doc=4028,freq=2.0), product of:
             0.1689437 = queryWeight, product of:
               1.090576 = boost
               6.541522 = idf(docFreq=166, maxDocs=42596)
               0.023681393 = queryNorm
             0.86729145 = fieldWeight in 4028, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.541522 = idf(docFreq=166, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.021592608 = weight(abstract_txt:that in 4028) [ClassicSimilarity], result of:
           0.021592608 = score(doc=4028,freq=2.0), product of:
             0.06797999 = queryWeight, product of:
               1.1982188 = boost
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.023681393 = queryNorm
             0.3176318 = fieldWeight in 4028, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.02252876 = weight(abstract_txt:information in 4028) [ClassicSimilarity], result of:
           0.02252876 = score(doc=4028,freq=2.0), product of:
             0.06993092 = queryWeight, product of:
               1.2152908 = boost
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.023681393 = queryNorm
             0.32215735 = fieldWeight in 4028, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.041294347 = weight(abstract_txt:have in 4028) [ClassicSimilarity], result of:
           0.041294347 = score(doc=4028,freq=2.0), product of:
             0.096321516 = queryWeight, product of:
               1.2578678 = boost
               3.233561 = idf(docFreq=4563, maxDocs=42596)
               0.023681393 = queryNorm
             0.42871365 = fieldWeight in 4028, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.233561 = idf(docFreq=4563, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.06544362 = weight(abstract_txt:query in 4028) [ClassicSimilarity], result of:
           0.06544362 = score(doc=4028,freq=1.0), product of:
             0.1474603 = queryWeight, product of:
               1.3153669 = boost
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.023681393 = queryNorm
             0.44380504 = fieldWeight in 4028, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.04914506 = weight(abstract_txt:text in 4028) [ClassicSimilarity], result of:
           0.04914506 = score(doc=4028,freq=1.0), product of:
             0.12946247 = queryWeight, product of:
               1.3501185 = boost
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.023681393 = queryNorm
             0.37960857 = fieldWeight in 4028, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.07985847 = weight(abstract_txt:retrieval in 4028) [ClassicSimilarity], result of:
           0.07985847 = score(doc=4028,freq=3.0), product of:
             0.14202385 = queryWeight, product of:
               1.7319136 = boost
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.023681393 = queryNorm
             0.5622891 = fieldWeight in 4028, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.14875118 = weight(abstract_txt:documents in 4028) [ClassicSimilarity], result of:
           0.14875118 = score(doc=4028,freq=3.0), product of:
             0.2226931 = queryWeight, product of:
               2.286008 = boost
               4.1135974 = idf(docFreq=1892, maxDocs=42596)
               0.023681393 = queryNorm
             0.66796494 = fieldWeight in 4028, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.1135974 = idf(docFreq=1892, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
         0.22759122 = weight(abstract_txt:words in 4028) [ClassicSimilarity], result of:
           0.22759122 = score(doc=4028,freq=1.0), product of:
             0.45318022 = queryWeight, product of:
               3.5723207 = boost
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.023681393 = queryNorm
             0.50220907 = fieldWeight in 4028, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.09375 = fieldNorm(doc=4028)
       0.4 = coord(10/25)
   

2. Vechtomova, O.; Karamuftuoglum, M.; Robertson, S.E.: On document relevance and lexical cohesion between query terms (2006) 0.33

   0.33062533 = sum of:
     0.33062533 = product of:
       0.9184036 = sum of:
         0.052951768 = weight(abstract_txt:semantic in 2167) [ClassicSimilarity], result of:
           0.052951768 = score(doc=2167,freq=2.0), product of:
             0.106534764 = queryWeight, product of:
               4.4986696 = idf(docFreq=1287, maxDocs=42596)
               0.023681393 = queryNorm
             0.49703747 = fieldWeight in 2167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4986696 = idf(docFreq=1287, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.24420573 = weight(abstract_txt:lexical in 2167) [ClassicSimilarity], result of:
           0.24420573 = score(doc=2167,freq=8.0), product of:
             0.1689437 = queryWeight, product of:
               1.090576 = boost
               6.541522 = idf(docFreq=166, maxDocs=42596)
               0.023681393 = queryNorm
             1.4454858 = fieldWeight in 2167, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               6.541522 = idf(docFreq=166, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.01799384 = weight(abstract_txt:that in 2167) [ClassicSimilarity], result of:
           0.01799384 = score(doc=2167,freq=2.0), product of:
             0.06797999 = queryWeight, product of:
               1.1982188 = boost
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.023681393 = queryNorm
             0.26469317 = fieldWeight in 2167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.0132752 = weight(abstract_txt:information in 2167) [ClassicSimilarity], result of:
           0.0132752 = score(doc=2167,freq=1.0), product of:
             0.06993092 = queryWeight, product of:
               1.2152908 = boost
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.023681393 = queryNorm
             0.18983305 = fieldWeight in 2167, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.09445973 = weight(abstract_txt:query in 2167) [ClassicSimilarity], result of:
           0.09445973 = score(doc=2167,freq=3.0), product of:
             0.1474603 = queryWeight, product of:
               1.3153669 = boost
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.023681393 = queryNorm
             0.6405774 = fieldWeight in 2167, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.081908435 = weight(abstract_txt:text in 2167) [ClassicSimilarity], result of:
           0.081908435 = score(doc=2167,freq=4.0), product of:
             0.12946247 = queryWeight, product of:
               1.3501185 = boost
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.023681393 = queryNorm
             0.63268095 = fieldWeight in 2167, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.038421925 = weight(abstract_txt:retrieval in 2167) [ClassicSimilarity], result of:
           0.038421925 = score(doc=2167,freq=1.0), product of:
             0.14202385 = queryWeight, product of:
               1.7319136 = boost
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.023681393 = queryNorm
             0.2705315 = fieldWeight in 2167, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.10696815 = weight(abstract_txt:terms in 2167) [ClassicSimilarity], result of:
           0.10696815 = score(doc=2167,freq=2.0), product of:
             0.2385164 = queryWeight, product of:
               2.481303 = boost
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.023681393 = queryNorm
             0.44847292 = fieldWeight in 2167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
         0.26821885 = weight(abstract_txt:words in 2167) [ClassicSimilarity], result of:
           0.26821885 = score(doc=2167,freq=2.0), product of:
             0.45318022 = queryWeight, product of:
               3.5723207 = boost
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.023681393 = queryNorm
             0.5918591 = fieldWeight in 2167, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.078125 = fieldNorm(doc=2167)
       0.36 = coord(9/25)
   

3. Aldana, J.F.; Gómez, A.C.; Moreno, N.; Nebro, A.J.; Roldán, M.M.: Metadata functionality for semantic Web integration (2003) 0.33

   0.32507706 = sum of:
     0.32507706 = product of:
       0.62514824 = sum of:
         0.05023446 = weight(abstract_txt:semantic in 3732) [ClassicSimilarity], result of:
           0.05023446 = score(doc=3732,freq=5.0), product of:
             0.106534764 = queryWeight, product of:
               4.4986696 = idf(docFreq=1287, maxDocs=42596)
               0.023681393 = queryNorm
             0.47153115 = fieldWeight in 3732, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.4986696 = idf(docFreq=1287, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.032774903 = weight(abstract_txt:among in 3732) [ClassicSimilarity], result of:
           0.032774903 = score(doc=3732,freq=2.0), product of:
             0.10876717 = queryWeight, product of:
               1.0104231 = boost
               4.5455594 = idf(docFreq=1228, maxDocs=42596)
               0.023681393 = queryNorm
             0.30133086 = fieldWeight in 3732, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.5455594 = idf(docFreq=1228, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.014238816 = weight(abstract_txt:used in 3732) [ClassicSimilarity], result of:
           0.014238816 = score(doc=3732,freq=1.0), product of:
             0.089982875 = queryWeight, product of:
               1.1255891 = boost
               3.375769 = idf(docFreq=3958, maxDocs=42596)
               0.023681393 = queryNorm
             0.15823917 = fieldWeight in 3732, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.375769 = idf(docFreq=3958, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.037970822 = weight(abstract_txt:vocabulary in 3732) [ClassicSimilarity], result of:
           0.037970822 = score(doc=3732,freq=1.0), product of:
             0.15116347 = queryWeight, product of:
               1.1911811 = boost
               5.35873 = idf(docFreq=544, maxDocs=42596)
               0.023681393 = queryNorm
             0.25119045 = fieldWeight in 3732, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.35873 = idf(docFreq=544, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.017070457 = weight(abstract_txt:that in 3732) [ClassicSimilarity], result of:
           0.017070457 = score(doc=3732,freq=5.0), product of:
             0.06797999 = queryWeight, product of:
               1.1982188 = boost
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.023681393 = queryNorm
             0.25111002 = fieldWeight in 3732, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.01781055 = weight(abstract_txt:information in 3732) [ClassicSimilarity], result of:
           0.01781055 = score(doc=3732,freq=5.0), product of:
             0.06993092 = queryWeight, product of:
               1.2152908 = boost
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.023681393 = queryNorm
             0.25468776 = fieldWeight in 3732, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.020647174 = weight(abstract_txt:have in 3732) [ClassicSimilarity], result of:
           0.020647174 = score(doc=3732,freq=2.0), product of:
             0.096321516 = queryWeight, product of:
               1.2578678 = boost
               3.233561 = idf(docFreq=4563, maxDocs=42596)
               0.023681393 = queryNorm
             0.21435682 = fieldWeight in 3732, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.233561 = idf(docFreq=4563, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.03272181 = weight(abstract_txt:query in 3732) [ClassicSimilarity], result of:
           0.03272181 = score(doc=3732,freq=1.0), product of:
             0.1474603 = queryWeight, product of:
               1.3153669 = boost
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.023681393 = queryNorm
             0.22190252 = fieldWeight in 3732, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.10102425 = weight(abstract_txt:irrelevant in 3732) [ClassicSimilarity], result of:
           0.10102425 = score(doc=3732,freq=1.0), product of:
             0.29024446 = queryWeight, product of:
               1.650579 = boost
               7.4254093 = idf(docFreq=68, maxDocs=42596)
               0.023681393 = queryNorm
             0.34806606 = fieldWeight in 3732, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.4254093 = idf(docFreq=68, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.032602083 = weight(abstract_txt:retrieval in 3732) [ClassicSimilarity], result of:
           0.032602083 = score(doc=3732,freq=2.0), product of:
             0.14202385 = queryWeight, product of:
               1.7319136 = boost
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.023681393 = queryNorm
             0.22955358 = fieldWeight in 3732, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.04294077 = weight(abstract_txt:documents in 3732) [ClassicSimilarity], result of:
           0.04294077 = score(doc=3732,freq=1.0), product of:
             0.2226931 = queryWeight, product of:
               2.286008 = boost
               4.1135974 = idf(docFreq=1892, maxDocs=42596)
               0.023681393 = queryNorm
             0.19282487 = fieldWeight in 3732, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1135974 = idf(docFreq=1892, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.06418089 = weight(abstract_txt:terms in 3732) [ClassicSimilarity], result of:
           0.06418089 = score(doc=3732,freq=2.0), product of:
             0.2385164 = queryWeight, product of:
               2.481303 = boost
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.023681393 = queryNorm
             0.26908374 = fieldWeight in 3732, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
         0.16093129 = weight(abstract_txt:words in 3732) [ClassicSimilarity], result of:
           0.16093129 = score(doc=3732,freq=2.0), product of:
             0.45318022 = queryWeight, product of:
               3.5723207 = boost
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.023681393 = queryNorm
             0.35511544 = fieldWeight in 3732, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.046875 = fieldNorm(doc=3732)
       0.52 = coord(13/25)
   

4. Jacquemin, C.: Spotting and discovering terms through natural language processing (2001) 0.32

   0.323941 = sum of:
     0.323941 = product of:
       0.73622954 = sum of:
         0.029954044 = weight(abstract_txt:semantic in 1120) [ClassicSimilarity], result of:
           0.029954044 = score(doc=1120,freq=1.0), product of:
             0.106534764 = queryWeight, product of:
               4.4986696 = idf(docFreq=1287, maxDocs=42596)
               0.023681393 = queryNorm
             0.28116685 = fieldWeight in 1120, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4986696 = idf(docFreq=1287, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.026848966 = weight(abstract_txt:used in 1120) [ClassicSimilarity], result of:
           0.026848966 = score(doc=1120,freq=2.0), product of:
             0.089982875 = queryWeight, product of:
               1.1255891 = boost
               3.375769 = idf(docFreq=3958, maxDocs=42596)
               0.023681393 = queryNorm
             0.29837862 = fieldWeight in 1120, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.375769 = idf(docFreq=3958, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.014395072 = weight(abstract_txt:that in 1120) [ClassicSimilarity], result of:
           0.014395072 = score(doc=1120,freq=2.0), product of:
             0.06797999 = queryWeight, product of:
               1.1982188 = boost
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.023681393 = queryNorm
             0.21175455 = fieldWeight in 1120, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.018394655 = weight(abstract_txt:information in 1120) [ClassicSimilarity], result of:
           0.018394655 = score(doc=1120,freq=3.0), product of:
             0.06993092 = queryWeight, product of:
               1.2152908 = boost
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.023681393 = queryNorm
             0.26304036 = fieldWeight in 1120, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.0600698 = weight(abstract_txt:texts in 1120) [ClassicSimilarity], result of:
           0.0600698 = score(doc=1120,freq=1.0), product of:
             0.16941759 = queryWeight, product of:
               1.2610537 = boost
               5.6730638 = idf(docFreq=397, maxDocs=42596)
               0.023681393 = queryNorm
             0.35456648 = fieldWeight in 1120, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6730638 = idf(docFreq=397, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.043629084 = weight(abstract_txt:query in 1120) [ClassicSimilarity], result of:
           0.043629084 = score(doc=1120,freq=1.0), product of:
             0.1474603 = queryWeight, product of:
               1.3153669 = boost
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.023681393 = queryNorm
             0.29587004 = fieldWeight in 1120, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7339206 = idf(docFreq=1017, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.04633441 = weight(abstract_txt:text in 1120) [ClassicSimilarity], result of:
           0.04633441 = score(doc=1120,freq=2.0), product of:
             0.12946247 = queryWeight, product of:
               1.3501185 = boost
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.023681393 = queryNorm
             0.35789838 = fieldWeight in 1120, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.053238977 = weight(abstract_txt:retrieval in 1120) [ClassicSimilarity], result of:
           0.053238977 = score(doc=1120,freq=3.0), product of:
             0.14202385 = queryWeight, product of:
               1.7319136 = boost
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.023681393 = queryNorm
             0.37485942 = fieldWeight in 1120, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.10776883 = weight(abstract_txt:controlled in 1120) [ClassicSimilarity], result of:
           0.10776883 = score(doc=1120,freq=1.0), product of:
             0.31515667 = queryWeight, product of:
               2.4323864 = boost
               5.4712515 = idf(docFreq=486, maxDocs=42596)
               0.023681393 = queryNorm
             0.34195322 = fieldWeight in 1120, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4712515 = idf(docFreq=486, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.121020645 = weight(abstract_txt:terms in 1120) [ClassicSimilarity], result of:
           0.121020645 = score(doc=1120,freq=4.0), product of:
             0.2385164 = queryWeight, product of:
               2.481303 = boost
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.023681393 = queryNorm
             0.5073892 = fieldWeight in 1120, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
         0.21457507 = weight(abstract_txt:words in 1120) [ClassicSimilarity], result of:
           0.21457507 = score(doc=1120,freq=2.0), product of:
             0.45318022 = queryWeight, product of:
               3.5723207 = boost
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.023681393 = queryNorm
             0.47348726 = fieldWeight in 1120, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.0625 = fieldNorm(doc=1120)
       0.44 = coord(11/25)
   

5. Wang, F.L.; Yang, C.C.: Mining Web data for Chinese segmentation (2007) 0.31

   0.31043658 = sum of:
     0.31043658 = product of:
       0.77609146 = sum of:
         0.014395072 = weight(abstract_txt:that in 1784) [ClassicSimilarity], result of:
           0.014395072 = score(doc=1784,freq=2.0), product of:
             0.06797999 = queryWeight, product of:
               1.1982188 = boost
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.023681393 = queryNorm
             0.21175455 = fieldWeight in 1784, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3957293 = idf(docFreq=10548, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.010620159 = weight(abstract_txt:information in 1784) [ClassicSimilarity], result of:
           0.010620159 = score(doc=1784,freq=1.0), product of:
             0.06993092 = queryWeight, product of:
               1.2152908 = boost
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.023681393 = queryNorm
             0.15186644 = fieldWeight in 1784, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.429863 = idf(docFreq=10194, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.05185294 = weight(abstract_txt:large in 1784) [ClassicSimilarity], result of:
           0.05185294 = score(doc=1784,freq=2.0), product of:
             0.13131973 = queryWeight, product of:
               1.2412931 = boost
               4.467334 = idf(docFreq=1328, maxDocs=42596)
               0.023681393 = queryNorm
             0.39486024 = fieldWeight in 1784, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.467334 = idf(docFreq=1328, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.033716694 = weight(abstract_txt:have in 1784) [ClassicSimilarity], result of:
           0.033716694 = score(doc=1784,freq=3.0), product of:
             0.096321516 = queryWeight, product of:
               1.2578678 = boost
               3.233561 = idf(docFreq=4563, maxDocs=42596)
               0.023681393 = queryNorm
             0.35004324 = fieldWeight in 1784, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.233561 = idf(docFreq=4563, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.08495152 = weight(abstract_txt:texts in 1784) [ClassicSimilarity], result of:
           0.08495152 = score(doc=1784,freq=2.0), product of:
             0.16941759 = queryWeight, product of:
               1.2610537 = boost
               5.6730638 = idf(docFreq=397, maxDocs=42596)
               0.023681393 = queryNorm
             0.5014327 = fieldWeight in 1784, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.6730638 = idf(docFreq=397, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.032763373 = weight(abstract_txt:text in 1784) [ClassicSimilarity], result of:
           0.032763373 = score(doc=1784,freq=1.0), product of:
             0.12946247 = queryWeight, product of:
               1.3501185 = boost
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.023681393 = queryNorm
             0.25307238 = fieldWeight in 1784, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.049158 = idf(docFreq=2018, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.030737538 = weight(abstract_txt:retrieval in 1784) [ClassicSimilarity], result of:
           0.030737538 = score(doc=1784,freq=1.0), product of:
             0.14202385 = queryWeight, product of:
               1.7319136 = boost
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.023681393 = queryNorm
             0.2164252 = fieldWeight in 1784, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4628031 = idf(docFreq=3628, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.12802464 = weight(abstract_txt:documents in 1784) [ClassicSimilarity], result of:
           0.12802464 = score(doc=1784,freq=5.0), product of:
             0.2226931 = queryWeight, product of:
               2.286008 = boost
               4.1135974 = idf(docFreq=1892, maxDocs=42596)
               0.023681393 = queryNorm
             0.5748927 = fieldWeight in 1784, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.1135974 = idf(docFreq=1892, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.085574515 = weight(abstract_txt:terms in 1784) [ClassicSimilarity], result of:
           0.085574515 = score(doc=1784,freq=2.0), product of:
             0.2385164 = queryWeight, product of:
               2.481303 = boost
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.023681393 = queryNorm
             0.35877833 = fieldWeight in 1784, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.0591135 = idf(docFreq=1998, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
         0.30345497 = weight(abstract_txt:words in 1784) [ClassicSimilarity], result of:
           0.30345497 = score(doc=1784,freq=4.0), product of:
             0.45318022 = queryWeight, product of:
               3.5723207 = boost
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.023681393 = queryNorm
             0.6696121 = fieldWeight in 1784, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.356897 = idf(docFreq=545, maxDocs=42596)
               0.0625 = fieldNorm(doc=1784)
       0.4 = coord(10/25)