Document (#30208)

Author

Khoo, C.S.G.
Dai, D.
Loh, T.E.

Title

Using statistical and contextual information to identify two- and three-character words in Chinese text

Source

Journal of the American Society for Information Science and technology. 53(2002) no.5, S.365-377

Year

2002

Abstract

Khoo, Dai, and Loh examine new statistical methods for the identification of two and three character words in Chinese text. Some meaningful Chinese words are simple (independent units of one or more characters in a sentence that have independent meaning) but others are compounds of two or more simple words. In their segmentation they utilize the Modern Chinese Word Segmentation for Application of Information Processing, with some modifications to focus on meaningful words to do manual segmentation. About 37% of meaningful words are longer than 2 characters indicating a need to handle three and four character words. Four hundred sentences from news articles were manually broken into overlapping bi-grams and tri-grams. Using logistic regression, the log of the odds that such bi/tri-grams were meaningful words was calculated. Variables like relative frequency, document frequency, local frequency, and contextual and positional information, were incorporated in the model only if the concordance measure improved by at least 2% with their addition. For two- and three-character words relative frequency of adjacent characters and document frequency of overlapping bi-grams were found to be significant. Using measures of recall and precision where correct automatic segmentation is normalized either by manual segmentation or by automatic segmentation, the contextual information formula for 2 character words provides significantly better results than previous formulations and using both the 2 and 3 character formulations in combination significantly improves the 2 character results.

Theme

Computerlinguistik

Similar documents (author)

1. Khoo, C.S.G.; Poo, D.C.C.: ¬An expert system approach to online catalog subject searching (1994) 6.11

   6.1113977 = sum of:
     6.1113977 = sum of:
       2.7258348 = weight(author_txt:khoo in 7303) [ClassicSimilarity], result of:
         2.7258348 = score(doc=7303,freq=1.0), product of:
           0.6544083 = queryWeight, product of:
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.07855395 = queryNorm
           4.165343 = fieldWeight in 7303, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.5 = fieldNorm(doc=7303)
       3.385563 = weight(author_txt:c.s.g in 7303) [ClassicSimilarity], result of:
         3.385563 = score(doc=7303,freq=1.0), product of:
           0.7561414 = queryWeight, product of:
             1.0749224 = boost
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.07855395 = queryNorm
           4.4774203 = fieldWeight in 7303, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.5 = fieldNorm(doc=7303)
   

2. Chaudhry, A.S.; Khoo, C.S.G..: ¬A survey of the top-level categories in the structure of corporate Websites (2008) 6.11

   6.1113977 = sum of:
     6.1113977 = sum of:
       2.7258348 = weight(author_txt:khoo in 4260) [ClassicSimilarity], result of:
         2.7258348 = score(doc=4260,freq=1.0), product of:
           0.6544083 = queryWeight, product of:
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.07855395 = queryNorm
           4.165343 = fieldWeight in 4260, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.5 = fieldNorm(doc=4260)
       3.385563 = weight(author_txt:c.s.g in 4260) [ClassicSimilarity], result of:
         3.385563 = score(doc=4260,freq=1.0), product of:
           0.7561414 = queryWeight, product of:
             1.0749224 = boost
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.07855395 = queryNorm
           4.4774203 = fieldWeight in 4260, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.5 = fieldNorm(doc=4260)
   

3. Khoo, C.S.G.; Ou, S.: Machine versus human clustering of concepts across documents (2008) 6.11

   6.1113977 = sum of:
     6.1113977 = sum of:
       2.7258348 = weight(author_txt:khoo in 4287) [ClassicSimilarity], result of:
         2.7258348 = score(doc=4287,freq=1.0), product of:
           0.6544083 = queryWeight, product of:
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.07855395 = queryNorm
           4.165343 = fieldWeight in 4287, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.5 = fieldNorm(doc=4287)
       3.385563 = weight(author_txt:c.s.g in 4287) [ClassicSimilarity], result of:
         3.385563 = score(doc=4287,freq=1.0), product of:
           0.7561414 = queryWeight, product of:
             1.0749224 = boost
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.07855395 = queryNorm
           4.4774203 = fieldWeight in 4287, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.5 = fieldNorm(doc=4287)
   

4. Poo, D.C.C.; Khoo, C.S.G.: Online Catalog Subject Searching (2009) 6.11

   6.1113977 = sum of:
     6.1113977 = sum of:
       2.7258348 = weight(author_txt:khoo in 852) [ClassicSimilarity], result of:
         2.7258348 = score(doc=852,freq=1.0), product of:
           0.6544083 = queryWeight, product of:
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.07855395 = queryNorm
           4.165343 = fieldWeight in 852, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.5 = fieldNorm(doc=852)
       3.385563 = weight(author_txt:c.s.g in 852) [ClassicSimilarity], result of:
         3.385563 = score(doc=852,freq=1.0), product of:
           0.7561414 = queryWeight, product of:
             1.0749224 = boost
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.07855395 = queryNorm
           4.4774203 = fieldWeight in 852, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.5 = fieldNorm(doc=852)
   

5. Sun, G.; Khoo, C.S.G.: ¬A framework to represent variables and values in social science research data sets to support data curation and reuse (2018) 6.11

   6.1113977 = sum of:
     6.1113977 = sum of:
       2.7258348 = weight(author_txt:khoo in 745) [ClassicSimilarity], result of:
         2.7258348 = score(doc=745,freq=1.0), product of:
           0.6544083 = queryWeight, product of:
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.07855395 = queryNorm
           4.165343 = fieldWeight in 745, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.330686 = idf(docFreq=27, maxDocs=42740)
             0.5 = fieldNorm(doc=745)
       3.385563 = weight(author_txt:c.s.g in 745) [ClassicSimilarity], result of:
         3.385563 = score(doc=745,freq=1.0), product of:
           0.7561414 = queryWeight, product of:
             1.0749224 = boost
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.07855395 = queryNorm
           4.4774203 = fieldWeight in 745, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.954841 = idf(docFreq=14, maxDocs=42740)
             0.5 = fieldNorm(doc=745)
   

Similar documents (content)

1. Yang, C.C.; Li, K.W.: ¬A heuristic method based on a statistical approach for chinese text segmentation (2005) 0.61

   0.6147954 = sum of:
     0.6147954 = product of:
       1.7077649 = sum of:
         0.055246923 = weight(abstract_txt:adjacent in 581) [ClassicSimilarity], result of:
           0.055246923 = score(doc=581,freq=1.0), product of:
             0.10011133 = queryWeight, product of:
               8.829678 = idf(docFreq=16, maxDocs=42740)
               0.0113380505 = queryNorm
             0.55185485 = fieldWeight in 581, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.829678 = idf(docFreq=16, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.0079793725 = weight(abstract_txt:information in 581) [ClassicSimilarity], result of:
           0.0079793725 = score(doc=581,freq=3.0), product of:
             0.030332152 = queryWeight, product of:
               1.10088 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0113380505 = queryNorm
             0.26306647 = fieldWeight in 581, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.022567257 = weight(abstract_txt:automatic in 581) [ClassicSimilarity], result of:
           0.022567257 = score(doc=581,freq=1.0), product of:
             0.06943957 = queryWeight, product of:
               1.1778145 = boost
               5.199861 = idf(docFreq=640, maxDocs=42740)
               0.0113380505 = queryNorm
             0.32499132 = fieldWeight in 581, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.199861 = idf(docFreq=640, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.03869676 = weight(abstract_txt:statistical in 581) [ClassicSimilarity], result of:
           0.03869676 = score(doc=581,freq=2.0), product of:
             0.078957655 = queryWeight, product of:
               1.2559446 = boost
               5.544793 = idf(docFreq=453, maxDocs=42740)
               0.0113380505 = queryNorm
             0.4900951 = fieldWeight in 581, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.544793 = idf(docFreq=453, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.095420465 = weight(abstract_txt:characters in 581) [ClassicSimilarity], result of:
           0.095420465 = score(doc=581,freq=1.0), product of:
             0.20784804 = queryWeight, product of:
               2.4956977 = boost
               7.3454022 = idf(docFreq=74, maxDocs=42740)
               0.0113380505 = queryNorm
             0.45908764 = fieldWeight in 581, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3454022 = idf(docFreq=74, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.24541327 = weight(abstract_txt:chinese in 581) [ClassicSimilarity], result of:
           0.24541327 = score(doc=581,freq=9.0), product of:
             0.20645039 = queryWeight, product of:
               2.872078 = boost
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0113380505 = queryNorm
             1.1887276 = fieldWeight in 581, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.24725084 = weight(abstract_txt:grams in 581) [ClassicSimilarity], result of:
           0.24725084 = score(doc=581,freq=2.0), product of:
             0.34254137 = queryWeight, product of:
               3.6995158 = boost
               8.166383 = idf(docFreq=32, maxDocs=42740)
               0.0113380505 = queryNorm
             0.7218131 = fieldWeight in 581, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               8.166383 = idf(docFreq=32, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.71908504 = weight(abstract_txt:segmentation in 581) [ClassicSimilarity], result of:
           0.71908504 = score(doc=581,freq=9.0), product of:
             0.48391333 = queryWeight, product of:
               5.385398 = boost
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0113380505 = queryNorm
             1.485979 = fieldWeight in 581, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
         0.2761049 = weight(abstract_txt:words in 581) [ClassicSimilarity], result of:
           0.2761049 = score(doc=581,freq=5.0), product of:
             0.36869782 = queryWeight, product of:
               6.068672 = boost
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0113380505 = queryNorm
             0.748865 = fieldWeight in 581, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0625 = fieldNorm(doc=581)
       0.36 = coord(9/25)
   

2. Wang, F.L.; Yang, C.C.: Mining Web data for Chinese segmentation (2007) 0.35

   0.3548715 = sum of:
     0.3548715 = product of:
       1.4786314 = sum of:
         0.004606893 = weight(abstract_txt:information in 2605) [ClassicSimilarity], result of:
           0.004606893 = score(doc=2605,freq=1.0), product of:
             0.030332152 = queryWeight, product of:
               1.10088 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0113380505 = queryNorm
             0.1518815 = fieldWeight in 2605, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0625 = fieldNorm(doc=2605)
         0.031726867 = weight(abstract_txt:independent in 2605) [ClassicSimilarity], result of:
           0.031726867 = score(doc=2605,freq=1.0), product of:
             0.08714436 = queryWeight, product of:
               1.3194503 = boost
               5.82516 = idf(docFreq=342, maxDocs=42740)
               0.0113380505 = queryNorm
             0.3640725 = fieldWeight in 2605, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.82516 = idf(docFreq=342, maxDocs=42740)
               0.0625 = fieldNorm(doc=2605)
         0.21643415 = weight(abstract_txt:chinese in 2605) [ClassicSimilarity], result of:
           0.21643415 = score(doc=2605,freq=7.0), product of:
             0.20645039 = queryWeight, product of:
               2.872078 = boost
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0113380505 = queryNorm
             1.0483592 = fieldWeight in 2605, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0625 = fieldNorm(doc=2605)
         0.22092545 = weight(abstract_txt:character in 2605) [ClassicSimilarity], result of:
           0.22092545 = score(doc=2605,freq=2.0), product of:
             0.38294026 = queryWeight, product of:
               5.174554 = boost
               6.527092 = idf(docFreq=169, maxDocs=42740)
               0.0113380505 = queryNorm
             0.57691884 = fieldWeight in 2605, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.527092 = idf(docFreq=169, maxDocs=42740)
               0.0625 = fieldNorm(doc=2605)
         0.7579822 = weight(abstract_txt:segmentation in 2605) [ClassicSimilarity], result of:
           0.7579822 = score(doc=2605,freq=10.0), product of:
             0.48391333 = queryWeight, product of:
               5.385398 = boost
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0113380505 = queryNorm
             1.5663594 = fieldWeight in 2605, product of:
               3.1622777 = tf(freq=10.0), with freq of:
                 10.0 = termFreq=10.0
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0625 = fieldNorm(doc=2605)
         0.24695572 = weight(abstract_txt:words in 2605) [ClassicSimilarity], result of:
           0.24695572 = score(doc=2605,freq=4.0), product of:
             0.36869782 = queryWeight, product of:
               6.068672 = boost
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0113380505 = queryNorm
             0.6698052 = fieldWeight in 2605, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0625 = fieldNorm(doc=2605)
       0.24 = coord(6/25)
   

3. Kwok, K.L.: Employing multiple representations for Chinese information retrieval (1999) 0.31

   0.31164998 = sum of:
     0.31164998 = product of:
       0.97390616 = sum of:
         0.004606893 = weight(abstract_txt:information in 4774) [ClassicSimilarity], result of:
           0.004606893 = score(doc=4774,freq=1.0), product of:
             0.030332152 = queryWeight, product of:
               1.10088 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0113380505 = queryNorm
             0.1518815 = fieldWeight in 4774, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.019124782 = weight(abstract_txt:using in 4774) [ClassicSimilarity], result of:
           0.019124782 = score(doc=4774,freq=2.0), product of:
             0.062184982 = queryWeight, product of:
               1.5762719 = boost
               3.4794931 = idf(docFreq=3580, maxDocs=42740)
               0.0113380505 = queryNorm
             0.30754665 = fieldWeight in 4774, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4794931 = idf(docFreq=3580, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.016157404 = weight(abstract_txt:were in 4774) [ClassicSimilarity], result of:
           0.016157404 = score(doc=4774,freq=1.0), product of:
             0.0700184 = queryWeight, product of:
               1.6726092 = boost
               3.69215 = idf(docFreq=2894, maxDocs=42740)
               0.0113380505 = queryNorm
             0.23075938 = fieldWeight in 4774, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.69215 = idf(docFreq=2894, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.13494492 = weight(abstract_txt:characters in 4774) [ClassicSimilarity], result of:
           0.13494492 = score(doc=4774,freq=2.0), product of:
             0.20784804 = queryWeight, product of:
               2.4956977 = boost
               7.3454022 = idf(docFreq=74, maxDocs=42740)
               0.0113380505 = queryNorm
             0.64924794 = fieldWeight in 4774, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.3454022 = idf(docFreq=74, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.11568893 = weight(abstract_txt:chinese in 4774) [ClassicSimilarity], result of:
           0.11568893 = score(doc=4774,freq=2.0), product of:
             0.20645039 = queryWeight, product of:
               2.872078 = boost
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0113380505 = queryNorm
             0.5603716 = fieldWeight in 4774, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.22092545 = weight(abstract_txt:character in 4774) [ClassicSimilarity], result of:
           0.22092545 = score(doc=4774,freq=2.0), product of:
             0.38294026 = queryWeight, product of:
               5.174554 = boost
               6.527092 = idf(docFreq=169, maxDocs=42740)
               0.0113380505 = queryNorm
             0.57691884 = fieldWeight in 4774, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.527092 = idf(docFreq=169, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.3389799 = weight(abstract_txt:segmentation in 4774) [ClassicSimilarity], result of:
           0.3389799 = score(doc=4774,freq=2.0), product of:
             0.48391333 = queryWeight, product of:
               5.385398 = boost
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0113380505 = queryNorm
             0.70049715 = fieldWeight in 4774, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
         0.12347786 = weight(abstract_txt:words in 4774) [ClassicSimilarity], result of:
           0.12347786 = score(doc=4774,freq=1.0), product of:
             0.36869782 = queryWeight, product of:
               6.068672 = boost
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0113380505 = queryNorm
             0.3349026 = fieldWeight in 4774, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0625 = fieldNorm(doc=4774)
       0.32 = coord(8/25)
   

4. Lee, K.H.; Ng, M.K.M.; Lu, Q.: Text segmentation for Chinese spell checking (1999) 0.30

   0.3032212 = sum of:
     0.3032212 = product of:
       1.2634217 = sum of:
         0.095420465 = weight(abstract_txt:characters in 4914) [ClassicSimilarity], result of:
           0.095420465 = score(doc=4914,freq=1.0), product of:
             0.20784804 = queryWeight, product of:
               2.4956977 = boost
               7.3454022 = idf(docFreq=74, maxDocs=42740)
               0.0113380505 = queryNorm
             0.45908764 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.3454022 = idf(docFreq=74, maxDocs=42740)
               0.0625 = fieldNorm(doc=4914)
         0.2003791 = weight(abstract_txt:chinese in 4914) [ClassicSimilarity], result of:
           0.2003791 = score(doc=4914,freq=6.0), product of:
             0.20645039 = queryWeight, product of:
               2.872078 = boost
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0113380505 = queryNorm
             0.970592 = fieldWeight in 4914, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0625 = fieldNorm(doc=4914)
         0.0850584 = weight(abstract_txt:frequency in 4914) [ClassicSimilarity], result of:
           0.0850584 = score(doc=4914,freq=1.0), product of:
             0.22825097 = queryWeight, product of:
               3.3763676 = boost
               5.962447 = idf(docFreq=298, maxDocs=42740)
               0.0113380505 = queryNorm
             0.37265295 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.962447 = idf(docFreq=298, maxDocs=42740)
               0.0625 = fieldNorm(doc=4914)
         0.15621789 = weight(abstract_txt:character in 4914) [ClassicSimilarity], result of:
           0.15621789 = score(doc=4914,freq=1.0), product of:
             0.38294026 = queryWeight, product of:
               5.174554 = boost
               6.527092 = idf(docFreq=169, maxDocs=42740)
               0.0113380505 = queryNorm
             0.40794325 = fieldWeight in 4914, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.527092 = idf(docFreq=169, maxDocs=42740)
               0.0625 = fieldNorm(doc=4914)
         0.47939003 = weight(abstract_txt:segmentation in 4914) [ClassicSimilarity], result of:
           0.47939003 = score(doc=4914,freq=4.0), product of:
             0.48391333 = queryWeight, product of:
               5.385398 = boost
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0113380505 = queryNorm
             0.9906526 = fieldWeight in 4914, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0625 = fieldNorm(doc=4914)
         0.24695572 = weight(abstract_txt:words in 4914) [ClassicSimilarity], result of:
           0.24695572 = score(doc=4914,freq=4.0), product of:
             0.36869782 = queryWeight, product of:
               6.068672 = boost
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0113380505 = queryNorm
             0.6698052 = fieldWeight in 4914, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.358442 = idf(docFreq=546, maxDocs=42740)
               0.0625 = fieldNorm(doc=4914)
       0.24 = coord(6/25)
   

5. Peng, F.; Huang, X.: Machine learning for Asian language text classification (2007) 0.26

   0.25640562 = sum of:
     0.25640562 = product of:
       0.91573435 = sum of:
         0.00651513 = weight(abstract_txt:information in 2832) [ClassicSimilarity], result of:
           0.00651513 = score(doc=2832,freq=2.0), product of:
             0.030332152 = queryWeight, product of:
               1.10088 = boost
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0113380505 = queryNorm
             0.21479288 = fieldWeight in 2832, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.430104 = idf(docFreq=10226, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
         0.02439675 = weight(abstract_txt:simple in 2832) [ClassicSimilarity], result of:
           0.02439675 = score(doc=2832,freq=1.0), product of:
             0.07314351 = queryWeight, product of:
               1.208819 = boost
               5.336741 = idf(docFreq=558, maxDocs=42740)
               0.0113380505 = queryNorm
             0.3335463 = fieldWeight in 2832, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.336741 = idf(docFreq=558, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
         0.027362742 = weight(abstract_txt:statistical in 2832) [ClassicSimilarity], result of:
           0.027362742 = score(doc=2832,freq=1.0), product of:
             0.078957655 = queryWeight, product of:
               1.2559446 = boost
               5.544793 = idf(docFreq=453, maxDocs=42740)
               0.0113380505 = queryNorm
             0.34654957 = fieldWeight in 2832, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.544793 = idf(docFreq=453, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
         0.027362742 = weight(abstract_txt:significantly in 2832) [ClassicSimilarity], result of:
           0.027362742 = score(doc=2832,freq=1.0), product of:
             0.078957655 = queryWeight, product of:
               1.2559446 = boost
               5.544793 = idf(docFreq=453, maxDocs=42740)
               0.0113380505 = queryNorm
             0.34654957 = fieldWeight in 2832, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.544793 = idf(docFreq=453, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
         0.032314807 = weight(abstract_txt:were in 2832) [ClassicSimilarity], result of:
           0.032314807 = score(doc=2832,freq=4.0), product of:
             0.0700184 = queryWeight, product of:
               1.6726092 = boost
               3.69215 = idf(docFreq=2894, maxDocs=42740)
               0.0113380505 = queryNorm
             0.46151876 = fieldWeight in 2832, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.69215 = idf(docFreq=2894, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
         0.16360885 = weight(abstract_txt:chinese in 2832) [ClassicSimilarity], result of:
           0.16360885 = score(doc=2832,freq=4.0), product of:
             0.20645039 = queryWeight, product of:
               2.872078 = boost
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0113380505 = queryNorm
             0.79248506 = fieldWeight in 2832, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.3398805 = idf(docFreq=204, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
         0.63417333 = weight(abstract_txt:segmentation in 2832) [ClassicSimilarity], result of:
           0.63417333 = score(doc=2832,freq=7.0), product of:
             0.48391333 = queryWeight, product of:
               5.385398 = boost
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0113380505 = queryNorm
             1.3105102 = fieldWeight in 2832, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               7.925221 = idf(docFreq=41, maxDocs=42740)
               0.0625 = fieldNorm(doc=2832)
       0.28 = coord(7/25)