Document (#28684)

Author
Kostoff, R.N.
Block, J.A.
Title
Factor matrix text filtering and clustering
Source
Journal of the American Society for Information Science and Technology. 56(2005) no.9, S.946-968
Year
2005
Abstract
The presence of trivial words in text databases can affect record or concept (words/phrases) clustering adversely. Additionally, the determination of whether a word/phrase is trivial is context-dependent. Our objective in the present article is to demonstrate a context-dependent trivial word filter to improve clustering quality. Factor analysis was used as a context-dependent trivial word filter for subsequent term clustering. Medline records for Raynaud's Phenomenon were used as the database, and words were extracted from the record abstracts. A factor matrix of these words was generated, and the words that had low factor loadings across all factors were identified, and eliminated. The remaining words, which had high factor loading values for at least one factor and therefore were influential in determining the theme of that factor, were input to the clustering algorithm. Both quantitative and qualitative analyses were used to show that factor matrix filtering leads to higher quality clusters and subsequent taxonomies.
Theme
Computerlinguistik

Similar documents (author)

  1. Kostoff, R.N.: Citation analysis cross field normalization : a new paradigm (1997) 2.20 
    2.1954656 = sum of:
  2.1954656 = product of:
    4.390931 = sum of:
      4.390931 = weight(author_txt:kostoff in 464) [ClassicSimilarity], result of:
        4.390931 = score(doc=464,freq=1.0), product of:
          0.7304943 = queryWeight, product of:
            1.0342461 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07344009 = queryNorm
          6.010904 = fieldWeight in 464, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=464)
    0.5 = coord(1/2)
  2. Kostoff, R.N.: ¬The use and misuse of citation analysis in research evaluation (1998) 2.20 
    2.1954656 = sum of:
  2.1954656 = product of:
    4.390931 = sum of:
      4.390931 = weight(author_txt:kostoff in 4129) [ClassicSimilarity], result of:
        4.390931 = score(doc=4129,freq=1.0), product of:
          0.7304943 = queryWeight, product of:
            1.0342461 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07344009 = queryNorm
          6.010904 = fieldWeight in 4129, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=4129)
    0.5 = coord(1/2)
  3. Kostoff, R.N.: Where is the research in the research literature? (2012) 2.20 
    2.1954656 = sum of:
  2.1954656 = product of:
    4.390931 = sum of:
      4.390931 = weight(author_txt:kostoff in 374) [ClassicSimilarity], result of:
        4.390931 = score(doc=374,freq=1.0), product of:
          0.7304943 = queryWeight, product of:
            1.0342461 = boost
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.07344009 = queryNorm
          6.010904 = fieldWeight in 374, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.617446 = idf(docFreq=7, maxDocs=44218)
            0.625 = fieldNorm(doc=374)
    0.5 = coord(1/2)
  4. Block, B.: Quality control makes WorldCat ever better (1999) 1.98 
    1.9845178 = sum of:
  1.9845178 = product of:
    3.9690356 = sum of:
      3.9690356 = weight(author_txt:block in 4398) [ClassicSimilarity], result of:
        3.9690356 = score(doc=4398,freq=1.0), product of:
          0.68291885 = queryWeight, product of:
            9.298992 = idf(docFreq=10, maxDocs=44218)
            0.07344009 = queryNorm
          5.81187 = fieldWeight in 4398, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.298992 = idf(docFreq=10, maxDocs=44218)
            0.625 = fieldNorm(doc=4398)
    0.5 = coord(1/2)
  5. Block, B.: Kooperative Neukatalogisierung : Eine neue Facette in der Zusammenlegung der Verbünde (2007) 1.98 
    1.9845178 = sum of:
  1.9845178 = product of:
    3.9690356 = sum of:
      3.9690356 = weight(author_txt:block in 6394) [ClassicSimilarity], result of:
        3.9690356 = score(doc=6394,freq=1.0), product of:
          0.68291885 = queryWeight, product of:
            9.298992 = idf(docFreq=10, maxDocs=44218)
            0.07344009 = queryNorm
          5.81187 = fieldWeight in 6394, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.298992 = idf(docFreq=10, maxDocs=44218)
            0.625 = fieldNorm(doc=6394)
    0.5 = coord(1/2)

Similar documents (content)

  1. Cai, X.; Li, W.: Enhancing sentence-level clustering with integrated and interactive frameworks for theme-based summarization (2011) 0.22 
    0.22440287 = sum of:
  0.22440287 = product of:
    0.8014388 = sum of:
      0.014461292 = weight(abstract_txt:text in 4770) [ClassicSimilarity], result of:
        0.014461292 = score(doc=4770,freq=1.0), product of:
          0.05721768 = queryWeight, product of:
            1.1619157 = boost
            4.0438666 = idf(docFreq=2106, maxDocs=44218)
            0.0121775195 = queryNorm
          0.25274166 = fieldWeight in 4770, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.0438666 = idf(docFreq=2106, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
      0.022028 = weight(abstract_txt:quality in 4770) [ClassicSimilarity], result of:
        0.022028 = score(doc=4770,freq=1.0), product of:
          0.075748764 = queryWeight, product of:
            1.3368949 = boost
            4.6528544 = idf(docFreq=1145, maxDocs=44218)
            0.0121775195 = queryNorm
          0.2908034 = fieldWeight in 4770, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.6528544 = idf(docFreq=1145, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
      0.012435234 = weight(abstract_txt:used in 4770) [ClassicSimilarity], result of:
        0.012435234 = score(doc=4770,freq=1.0), product of:
          0.059227753 = queryWeight, product of:
            1.4478306 = boost
            3.3592992 = idf(docFreq=4177, maxDocs=44218)
            0.0121775195 = queryNorm
          0.2099562 = fieldWeight in 4770, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.3592992 = idf(docFreq=4177, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
      0.037921265 = weight(abstract_txt:context in 4770) [ClassicSimilarity], result of:
        0.037921265 = score(doc=4770,freq=2.0), product of:
          0.09885557 = queryWeight, product of:
            1.8704913 = boost
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0121775195 = queryNorm
          0.3836027 = fieldWeight in 4770, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
      0.105349235 = weight(abstract_txt:word in 4770) [ClassicSimilarity], result of:
        0.105349235 = score(doc=4770,freq=4.0), product of:
          0.15505658 = queryWeight, product of:
            2.342611 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0121775195 = queryNorm
          0.67942446 = fieldWeight in 4770, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
      0.50861293 = weight(abstract_txt:clustering in 4770) [ClassicSimilarity], result of:
        0.50861293 = score(doc=4770,freq=15.0), product of:
          0.33801296 = queryWeight, product of:
            4.465254 = boost
            6.2162485 = idf(docFreq=239, maxDocs=44218)
            0.0121775195 = queryNorm
          1.5047143 = fieldWeight in 4770, product of:
            3.8729835 = tf(freq=15.0), with freq of:
              15.0 = termFreq=15.0
            6.2162485 = idf(docFreq=239, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
      0.100630894 = weight(abstract_txt:words in 4770) [ClassicSimilarity], result of:
        0.100630894 = score(doc=4770,freq=1.0), product of:
          0.30078316 = queryWeight, product of:
            4.614205 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0121775195 = queryNorm
          0.33456293 = fieldWeight in 4770, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0625 = fieldNorm(doc=4770)
    0.28 = coord(7/25)
  2. Kalish, M.: Immersion multimedia for adult Chiricahua language learners (2005) 0.17 
    0.16690616 = sum of:
  0.16690616 = product of:
    0.5960934 = sum of:
      0.0054547437 = weight(abstract_txt:that in 5916) [ClassicSimilarity], result of:
        0.0054547437 = score(doc=5916,freq=1.0), product of:
          0.029466745 = queryWeight, product of:
            1.0212244 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0121775195 = queryNorm
          0.18511525 = fieldWeight in 5916, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
      0.018076615 = weight(abstract_txt:text in 5916) [ClassicSimilarity], result of:
        0.018076615 = score(doc=5916,freq=1.0), product of:
          0.05721768 = queryWeight, product of:
            1.1619157 = boost
            4.0438666 = idf(docFreq=2106, maxDocs=44218)
            0.0121775195 = queryNorm
          0.3159271 = fieldWeight in 5916, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.0438666 = idf(docFreq=2106, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
      0.04740158 = weight(abstract_txt:context in 5916) [ClassicSimilarity], result of:
        0.04740158 = score(doc=5916,freq=2.0), product of:
          0.09885557 = queryWeight, product of:
            1.8704913 = boost
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0121775195 = queryNorm
          0.4795034 = fieldWeight in 5916, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
      0.09311645 = weight(abstract_txt:word in 5916) [ClassicSimilarity], result of:
        0.09311645 = score(doc=5916,freq=2.0), product of:
          0.15505658 = queryWeight, product of:
            2.342611 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0121775195 = queryNorm
          0.60053205 = fieldWeight in 5916, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
      0.040538594 = weight(abstract_txt:were in 5916) [ClassicSimilarity], result of:
        0.040538594 = score(doc=5916,freq=1.0), product of:
          0.14138556 = queryWeight, product of:
            3.1635349 = boost
            3.6700637 = idf(docFreq=3061, maxDocs=44218)
            0.0121775195 = queryNorm
          0.28672373 = fieldWeight in 5916, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.6700637 = idf(docFreq=3061, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
      0.17789197 = weight(abstract_txt:words in 5916) [ClassicSimilarity], result of:
        0.17789197 = score(doc=5916,freq=2.0), product of:
          0.30078316 = queryWeight, product of:
            4.614205 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0121775195 = queryNorm
          0.5914293 = fieldWeight in 5916, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
      0.21361347 = weight(abstract_txt:factor in 5916) [ClassicSimilarity], result of:
        0.21361347 = score(doc=5916,freq=1.0), product of:
          0.47122097 = queryWeight, product of:
            6.668863 = boost
            5.8024845 = idf(docFreq=362, maxDocs=44218)
            0.0121775195 = queryNorm
          0.4533191 = fieldWeight in 5916, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.8024845 = idf(docFreq=362, maxDocs=44218)
            0.078125 = fieldNorm(doc=5916)
    0.28 = coord(7/25)
  3. Leydesdorff, L.; Zhou, P.: Co-word analysis using the Chinese character set (2008) 0.15 
    0.15430853 = sum of:
  0.15430853 = product of:
    0.7715426 = sum of:
      0.006545692 = weight(abstract_txt:that in 1970) [ClassicSimilarity], result of:
        0.006545692 = score(doc=1970,freq=1.0), product of:
          0.029466745 = queryWeight, product of:
            1.0212244 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0121775195 = queryNorm
          0.22213829 = fieldWeight in 1970, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.09375 = fieldNorm(doc=1970)
      0.13685267 = weight(abstract_txt:word in 1970) [ClassicSimilarity], result of:
        0.13685267 = score(doc=1970,freq=3.0), product of:
          0.15505658 = queryWeight, product of:
            2.342611 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0121775195 = queryNorm
          0.8825982 = fieldWeight in 1970, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.09375 = fieldNorm(doc=1970)
      0.15833773 = weight(abstract_txt:matrix in 1970) [ClassicSimilarity], result of:
        0.15833773 = score(doc=1970,freq=1.0), product of:
          0.24646279 = queryWeight, product of:
            2.953458 = boost
            6.8527 = idf(docFreq=126, maxDocs=44218)
            0.0121775195 = queryNorm
          0.6424407 = fieldWeight in 1970, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.8527 = idf(docFreq=126, maxDocs=44218)
            0.09375 = fieldNorm(doc=1970)
      0.21347035 = weight(abstract_txt:words in 1970) [ClassicSimilarity], result of:
        0.21347035 = score(doc=1970,freq=2.0), product of:
          0.30078316 = queryWeight, product of:
            4.614205 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0121775195 = queryNorm
          0.7097151 = fieldWeight in 1970, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.09375 = fieldNorm(doc=1970)
      0.25633615 = weight(abstract_txt:factor in 1970) [ClassicSimilarity], result of:
        0.25633615 = score(doc=1970,freq=1.0), product of:
          0.47122097 = queryWeight, product of:
            6.668863 = boost
            5.8024845 = idf(docFreq=362, maxDocs=44218)
            0.0121775195 = queryNorm
          0.5439829 = fieldWeight in 1970, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.8024845 = idf(docFreq=362, maxDocs=44218)
            0.09375 = fieldNorm(doc=1970)
    0.2 = coord(5/25)
  4. Lund, K.; Burgess, C.; Atchley, R.A.: Semantic and associative priming in high-dimensional semantic space (1995) 0.15 
    0.14710498 = sum of:
  0.14710498 = product of:
    0.7355249 = sum of:
      0.010799841 = weight(abstract_txt:that in 2151) [ClassicSimilarity], result of:
        0.010799841 = score(doc=2151,freq=2.0), product of:
          0.029466745 = queryWeight, product of:
            1.0212244 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0121775195 = queryNorm
          0.36650947 = fieldWeight in 2151, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.109375 = fieldNorm(doc=2151)
      0.13036302 = weight(abstract_txt:word in 2151) [ClassicSimilarity], result of:
        0.13036302 = score(doc=2151,freq=2.0), product of:
          0.15505658 = queryWeight, product of:
            2.342611 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0121775195 = queryNorm
          0.84074485 = fieldWeight in 2151, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.109375 = fieldNorm(doc=2151)
      0.3199571 = weight(abstract_txt:matrix in 2151) [ClassicSimilarity], result of:
        0.3199571 = score(doc=2151,freq=3.0), product of:
          0.24646279 = queryWeight, product of:
            2.953458 = boost
            6.8527 = idf(docFreq=126, maxDocs=44218)
            0.0121775195 = queryNorm
          1.2981964 = fieldWeight in 2151, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            6.8527 = idf(docFreq=126, maxDocs=44218)
            0.109375 = fieldNorm(doc=2151)
      0.09830087 = weight(abstract_txt:were in 2151) [ClassicSimilarity], result of:
        0.09830087 = score(doc=2151,freq=3.0), product of:
          0.14138556 = queryWeight, product of:
            3.1635349 = boost
            3.6700637 = idf(docFreq=3061, maxDocs=44218)
            0.0121775195 = queryNorm
          0.6952681 = fieldWeight in 2151, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            3.6700637 = idf(docFreq=3061, maxDocs=44218)
            0.109375 = fieldNorm(doc=2151)
      0.17610405 = weight(abstract_txt:words in 2151) [ClassicSimilarity], result of:
        0.17610405 = score(doc=2151,freq=1.0), product of:
          0.30078316 = queryWeight, product of:
            4.614205 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0121775195 = queryNorm
          0.5854851 = fieldWeight in 2151, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.109375 = fieldNorm(doc=2151)
    0.2 = coord(5/25)
  5. Lee, G.E.; Sun, A.: Understanding the stability of medical concept embeddings (2021) 0.14 
    0.14252478 = sum of:
  0.14252478 = product of:
    0.59385324 = sum of:
      0.004363795 = weight(abstract_txt:that in 159) [ClassicSimilarity], result of:
        0.004363795 = score(doc=159,freq=1.0), product of:
          0.029466745 = queryWeight, product of:
            1.0212244 = boost
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0121775195 = queryNorm
          0.1480922 = fieldWeight in 159, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.3694751 = idf(docFreq=11241, maxDocs=44218)
            0.0625 = fieldNorm(doc=159)
      0.022028 = weight(abstract_txt:quality in 159) [ClassicSimilarity], result of:
        0.022028 = score(doc=159,freq=1.0), product of:
          0.075748764 = queryWeight, product of:
            1.3368949 = boost
            4.6528544 = idf(docFreq=1145, maxDocs=44218)
            0.0121775195 = queryNorm
          0.2908034 = fieldWeight in 159, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.6528544 = idf(docFreq=1145, maxDocs=44218)
            0.0625 = fieldNorm(doc=159)
      0.037921265 = weight(abstract_txt:context in 159) [ClassicSimilarity], result of:
        0.037921265 = score(doc=159,freq=2.0), product of:
          0.09885557 = queryWeight, product of:
            1.8704913 = boost
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0121775195 = queryNorm
          0.3836027 = fieldWeight in 159, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0625 = fieldNorm(doc=159)
      0.09123511 = weight(abstract_txt:word in 159) [ClassicSimilarity], result of:
        0.09123511 = score(doc=159,freq=3.0), product of:
          0.15505658 = queryWeight, product of:
            2.342611 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0121775195 = queryNorm
          0.5883988 = fieldWeight in 159, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0625 = fieldNorm(doc=159)
      0.14231357 = weight(abstract_txt:words in 159) [ClassicSimilarity], result of:
        0.14231357 = score(doc=159,freq=2.0), product of:
          0.30078316 = queryWeight, product of:
            4.614205 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0121775195 = queryNorm
          0.47314343 = fieldWeight in 159, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0625 = fieldNorm(doc=159)
      0.29599148 = weight(abstract_txt:factor in 159) [ClassicSimilarity], result of:
        0.29599148 = score(doc=159,freq=3.0), product of:
          0.47122097 = queryWeight, product of:
            6.668863 = boost
            5.8024845 = idf(docFreq=362, maxDocs=44218)
            0.0121775195 = queryNorm
          0.62813735 = fieldWeight in 159, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.8024845 = idf(docFreq=362, maxDocs=44218)
            0.0625 = fieldNorm(doc=159)
    0.24 = coord(6/25)