Document (#28682)

Author
Kostoff, R.N.
Block, J.A.
Title
Factor matrix text filtering and clustering
Source
Journal of the American Society for Information Science and Technology. 56(2005) no.9, S.946-968
Year
2005
Abstract
The presence of trivial words in text databases can affect record or concept (words/phrases) clustering adversely. Additionally, the determination of whether a word/phrase is trivial is context-dependent. Our objective in the present article is to demonstrate a context-dependent trivial word filter to improve clustering quality. Factor analysis was used as a context-dependent trivial word filter for subsequent term clustering. Medline records for Raynaud's Phenomenon were used as the database, and words were extracted from the record abstracts. A factor matrix of these words was generated, and the words that had low factor loadings across all factors were identified, and eliminated. The remaining words, which had high factor loading values for at least one factor and therefore were influential in determining the theme of that factor, were input to the clustering algorithm. Both quantitative and qualitative analyses were used to show that factor matrix filtering leads to higher quality clusters and subsequent taxonomies.
Theme
Computerlinguistik

Similar documents (author)

  1. Kostoff, R.N.: Citation analysis cross field normalization : a new paradigm (1997) 2.19
    2.192132 = sum of:
      2.192132 = product of:
        4.384264 = sum of:
          4.384264 = weight(author_txt:kostoff in 1462) [ClassicSimilarity], result of:
            4.384264 = score(doc=1462,freq=1.0), product of:
              0.7305309 = queryWeight, product of:
                1.0343016 = boost
                9.602362 = idf(docFreq=7, maxDocs=43556)
                0.07355519 = queryNorm
              6.0014763 = fieldWeight in 1462, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.602362 = idf(docFreq=7, maxDocs=43556)
                0.625 = fieldNorm(doc=1462)
        0.5 = coord(1/2)
    
  2. Kostoff, R.N.: ¬The use and misuse of citation analysis in research evaluation (1998) 2.19
    2.192132 = sum of:
      2.192132 = product of:
        4.384264 = sum of:
          4.384264 = weight(author_txt:kostoff in 127) [ClassicSimilarity], result of:
            4.384264 = score(doc=127,freq=1.0), product of:
              0.7305309 = queryWeight, product of:
                1.0343016 = boost
                9.602362 = idf(docFreq=7, maxDocs=43556)
                0.07355519 = queryNorm
              6.0014763 = fieldWeight in 127, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.602362 = idf(docFreq=7, maxDocs=43556)
                0.625 = fieldNorm(doc=127)
        0.5 = coord(1/2)
    
  3. Kostoff, R.N.: Where is the research in the research literature? (2012) 2.19
    2.192132 = sum of:
      2.192132 = product of:
        4.384264 = sum of:
          4.384264 = weight(author_txt:kostoff in 2372) [ClassicSimilarity], result of:
            4.384264 = score(doc=2372,freq=1.0), product of:
              0.7305309 = queryWeight, product of:
                1.0343016 = boost
                9.602362 = idf(docFreq=7, maxDocs=43556)
                0.07355519 = queryNorm
              6.0014763 = fieldWeight in 2372, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.602362 = idf(docFreq=7, maxDocs=43556)
                0.625 = fieldNorm(doc=2372)
        0.5 = coord(1/2)
    
  4. Block, B.: Quality control makes WorldCat ever better (1999) 1.98
    1.9811847 = sum of:
      1.9811847 = product of:
        3.9623694 = sum of:
          3.9623694 = weight(author_txt:block in 5396) [ClassicSimilarity], result of:
            3.9623694 = score(doc=5396,freq=1.0), product of:
              0.68287957 = queryWeight, product of:
                9.283908 = idf(docFreq=10, maxDocs=43556)
                0.07355519 = queryNorm
              5.8024426 = fieldWeight in 5396, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.283908 = idf(docFreq=10, maxDocs=43556)
                0.625 = fieldNorm(doc=5396)
        0.5 = coord(1/2)
    
  5. Block, B.: Kooperative Neukatalogisierung : Eine neue Facette in der Zusammenlegung der Verbünde (2007) 1.98
    1.9811847 = sum of:
      1.9811847 = product of:
        3.9623694 = sum of:
          3.9623694 = weight(author_txt:block in 392) [ClassicSimilarity], result of:
            3.9623694 = score(doc=392,freq=1.0), product of:
              0.68287957 = queryWeight, product of:
                9.283908 = idf(docFreq=10, maxDocs=43556)
                0.07355519 = queryNorm
              5.8024426 = fieldWeight in 392, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.283908 = idf(docFreq=10, maxDocs=43556)
                0.625 = fieldNorm(doc=392)
        0.5 = coord(1/2)
    

Similar documents (content)

  1. Cai, X.; Li, W.: Enhancing sentence-level clustering with integrated and interactive frameworks for theme-based summarization (2011) 0.23
    0.22580233 = sum of:
      0.22580233 = product of:
        0.8064369 = sum of:
          0.014506681 = weight(abstract_txt:text in 1768) [ClassicSimilarity], result of:
            0.014506681 = score(doc=1768,freq=1.0), product of:
              0.05731881 = queryWeight, product of:
                1.1600673 = boost
                4.0494018 = idf(docFreq=2063, maxDocs=43556)
                0.012201779 = queryNorm
              0.2530876 = fieldWeight in 1768, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.0494018 = idf(docFreq=2063, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
          0.02232391 = weight(abstract_txt:quality in 1768) [ClassicSimilarity], result of:
            0.02232391 = score(doc=1768,freq=1.0), product of:
              0.07640093 = queryWeight, product of:
                1.3393186 = boost
                4.6751075 = idf(docFreq=1103, maxDocs=43556)
                0.012201779 = queryNorm
              0.29219422 = fieldWeight in 1768, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6751075 = idf(docFreq=1103, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
          0.012497576 = weight(abstract_txt:used in 1768) [ClassicSimilarity], result of:
            0.012497576 = score(doc=1768,freq=1.0), product of:
              0.059406333 = queryWeight, product of:
                1.4464272 = boost
                3.3659916 = idf(docFreq=4087, maxDocs=43556)
                0.012201779 = queryNorm
              0.21037447 = fieldWeight in 1768, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.3659916 = idf(docFreq=4087, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
          0.03828839 = weight(abstract_txt:context in 1768) [ClassicSimilarity], result of:
            0.03828839 = score(doc=1768,freq=2.0), product of:
              0.09946042 = queryWeight, product of:
                1.8715669 = boost
                4.355337 = idf(docFreq=1519, maxDocs=43556)
                0.012201779 = queryNorm
              0.38496104 = fieldWeight in 1768, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.355337 = idf(docFreq=1519, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
          0.10549035 = weight(abstract_txt:word in 1768) [ClassicSimilarity], result of:
            0.10549035 = score(doc=1768,freq=4.0), product of:
              0.15514486 = queryWeight, product of:
                2.3374853 = boost
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.012201779 = queryNorm
              0.67994744 = fieldWeight in 1768, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
          0.51274824 = weight(abstract_txt:clustering in 1768) [ClassicSimilarity], result of:
            0.51274824 = score(doc=1768,freq=15.0), product of:
              0.3397328 = queryWeight, product of:
                4.4655333 = boost
                6.2350655 = idf(docFreq=231, maxDocs=43556)
                0.012201779 = queryNorm
              1.5092691 = fieldWeight in 1768, product of:
                3.8729835 = tf(freq=15.0), with freq of:
                  15.0 = termFreq=15.0
                6.2350655 = idf(docFreq=231, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
          0.10058176 = weight(abstract_txt:words in 1768) [ClassicSimilarity], result of:
            0.10058176 = score(doc=1768,freq=1.0), product of:
              0.30058804 = queryWeight, product of:
                4.6013055 = boost
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.012201779 = queryNorm
              0.33461663 = fieldWeight in 1768, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.0625 = fieldNorm(doc=1768)
        0.28 = coord(7/25)
    
  2. Kalish, M.: Immersion multimedia for adult Chiricahua language learners (2005) 0.17
    0.1673368 = sum of:
      0.1673368 = product of:
        0.59763145 = sum of:
          0.005534331 = weight(abstract_txt:that in 914) [ClassicSimilarity], result of:
            0.005534331 = score(doc=914,freq=1.0), product of:
              0.02974306 = queryWeight, product of:
                1.0234652 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.012201779 = queryNorm
              0.18607134 = fieldWeight in 914, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
          0.018133352 = weight(abstract_txt:text in 914) [ClassicSimilarity], result of:
            0.018133352 = score(doc=914,freq=1.0), product of:
              0.05731881 = queryWeight, product of:
                1.1600673 = boost
                4.0494018 = idf(docFreq=2063, maxDocs=43556)
                0.012201779 = queryNorm
              0.31635952 = fieldWeight in 914, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.0494018 = idf(docFreq=2063, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
          0.047860485 = weight(abstract_txt:context in 914) [ClassicSimilarity], result of:
            0.047860485 = score(doc=914,freq=2.0), product of:
              0.09946042 = queryWeight, product of:
                1.8715669 = boost
                4.355337 = idf(docFreq=1519, maxDocs=43556)
                0.012201779 = queryNorm
              0.4812013 = fieldWeight in 914, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.355337 = idf(docFreq=1519, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
          0.09324117 = weight(abstract_txt:word in 914) [ClassicSimilarity], result of:
            0.09324117 = score(doc=914,freq=2.0), product of:
              0.15514486 = queryWeight, product of:
                2.3374853 = boost
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.012201779 = queryNorm
              0.6009943 = fieldWeight in 914, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
          0.040855195 = weight(abstract_txt:were in 914) [ClassicSimilarity], result of:
            0.040855195 = score(doc=914,freq=1.0), product of:
              0.14207481 = queryWeight, product of:
                3.1633976 = boost
                3.6807828 = idf(docFreq=2983, maxDocs=43556)
                0.012201779 = queryNorm
              0.28756115 = fieldWeight in 914, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6807828 = idf(docFreq=2983, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
          0.1778051 = weight(abstract_txt:words in 914) [ClassicSimilarity], result of:
            0.1778051 = score(doc=914,freq=2.0), product of:
              0.30058804 = queryWeight, product of:
                4.6013055 = boost
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.012201779 = queryNorm
              0.5915242 = fieldWeight in 914, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
          0.21420182 = weight(abstract_txt:factor in 914) [ClassicSimilarity], result of:
            0.21420182 = score(doc=914,freq=1.0), product of:
              0.47193325 = queryWeight, product of:
                6.6574035 = boost
                5.8096848 = idf(docFreq=354, maxDocs=43556)
                0.012201779 = queryNorm
              0.45388162 = fieldWeight in 914, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.8096848 = idf(docFreq=354, maxDocs=43556)
                0.078125 = fieldNorm(doc=914)
        0.28 = coord(7/25)
    
  3. Leydesdorff, L.; Zhou, P.: Co-word analysis using the Chinese character set (2008) 0.15
    0.15446487 = sum of:
      0.15446487 = product of:
        0.7723243 = sum of:
          0.0066411975 = weight(abstract_txt:that in 3968) [ClassicSimilarity], result of:
            0.0066411975 = score(doc=3968,freq=1.0), product of:
              0.02974306 = queryWeight, product of:
                1.0234652 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.012201779 = queryNorm
              0.22328562 = fieldWeight in 3968, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.09375 = fieldNorm(doc=3968)
          0.13703598 = weight(abstract_txt:word in 3968) [ClassicSimilarity], result of:
            0.13703598 = score(doc=3968,freq=3.0), product of:
              0.15514486 = queryWeight, product of:
                2.3374853 = boost
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.012201779 = queryNorm
              0.88327765 = fieldWeight in 3968, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.09375 = fieldNorm(doc=3968)
          0.15823886 = weight(abstract_txt:matrix in 3968) [ClassicSimilarity], result of:
            0.15823886 = score(doc=3968,freq=1.0), product of:
              0.24628057 = queryWeight, product of:
                2.9450676 = boost
                6.853489 = idf(docFreq=124, maxDocs=43556)
                0.012201779 = queryNorm
              0.6425146 = fieldWeight in 3968, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.853489 = idf(docFreq=124, maxDocs=43556)
                0.09375 = fieldNorm(doc=3968)
          0.21336612 = weight(abstract_txt:words in 3968) [ClassicSimilarity], result of:
            0.21336612 = score(doc=3968,freq=2.0), product of:
              0.30058804 = queryWeight, product of:
                4.6013055 = boost
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.012201779 = queryNorm
              0.70982903 = fieldWeight in 3968, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.09375 = fieldNorm(doc=3968)
          0.2570422 = weight(abstract_txt:factor in 3968) [ClassicSimilarity], result of:
            0.2570422 = score(doc=3968,freq=1.0), product of:
              0.47193325 = queryWeight, product of:
                6.6574035 = boost
                5.8096848 = idf(docFreq=354, maxDocs=43556)
                0.012201779 = queryNorm
              0.54465795 = fieldWeight in 3968, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.8096848 = idf(docFreq=354, maxDocs=43556)
                0.09375 = fieldNorm(doc=3968)
        0.2 = coord(5/25)
    
  4. Lund, K.; Burgess, C.; Atchley, R.A.: Semantic and associative priming in high-dimensional semantic space (1995) 0.15
    0.14726782 = sum of:
      0.14726782 = product of:
        0.7363391 = sum of:
          0.010957416 = weight(abstract_txt:that in 4149) [ClassicSimilarity], result of:
            0.010957416 = score(doc=4149,freq=2.0), product of:
              0.02974306 = queryWeight, product of:
                1.0234652 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.012201779 = queryNorm
              0.36840245 = fieldWeight in 4149, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.109375 = fieldNorm(doc=4149)
          0.13053764 = weight(abstract_txt:word in 4149) [ClassicSimilarity], result of:
            0.13053764 = score(doc=4149,freq=2.0), product of:
              0.15514486 = queryWeight, product of:
                2.3374853 = boost
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.012201779 = queryNorm
              0.84139204 = fieldWeight in 4149, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.109375 = fieldNorm(doc=4149)
          0.31975737 = weight(abstract_txt:matrix in 4149) [ClassicSimilarity], result of:
            0.31975737 = score(doc=4149,freq=3.0), product of:
              0.24628057 = queryWeight, product of:
                2.9450676 = boost
                6.853489 = idf(docFreq=124, maxDocs=43556)
                0.012201779 = queryNorm
              1.2983459 = fieldWeight in 4149, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.853489 = idf(docFreq=124, maxDocs=43556)
                0.109375 = fieldNorm(doc=4149)
          0.09906859 = weight(abstract_txt:were in 4149) [ClassicSimilarity], result of:
            0.09906859 = score(doc=4149,freq=3.0), product of:
              0.14207481 = queryWeight, product of:
                3.1633976 = boost
                3.6807828 = idf(docFreq=2983, maxDocs=43556)
                0.012201779 = queryNorm
              0.69729877 = fieldWeight in 4149, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                3.6807828 = idf(docFreq=2983, maxDocs=43556)
                0.109375 = fieldNorm(doc=4149)
          0.17601807 = weight(abstract_txt:words in 4149) [ClassicSimilarity], result of:
            0.17601807 = score(doc=4149,freq=1.0), product of:
              0.30058804 = queryWeight, product of:
                4.6013055 = boost
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.012201779 = queryNorm
              0.5855791 = fieldWeight in 4149, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.109375 = fieldNorm(doc=4149)
        0.2 = coord(5/25)
    
  5. Lee, G.E.; Sun, A.: Understanding the stability of medical concept embeddings (2021) 0.14
    0.14290749 = sum of:
      0.14290749 = product of:
        0.5954479 = sum of:
          0.0044274647 = weight(abstract_txt:that in 2446) [ClassicSimilarity], result of:
            0.0044274647 = score(doc=2446,freq=1.0), product of:
              0.02974306 = queryWeight, product of:
                1.0234652 = boost
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.012201779 = queryNorm
              0.14885707 = fieldWeight in 2446, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                2.3817132 = idf(docFreq=10938, maxDocs=43556)
                0.0625 = fieldNorm(doc=2446)
          0.02232391 = weight(abstract_txt:quality in 2446) [ClassicSimilarity], result of:
            0.02232391 = score(doc=2446,freq=1.0), product of:
              0.07640093 = queryWeight, product of:
                1.3393186 = boost
                4.6751075 = idf(docFreq=1103, maxDocs=43556)
                0.012201779 = queryNorm
              0.29219422 = fieldWeight in 2446, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6751075 = idf(docFreq=1103, maxDocs=43556)
                0.0625 = fieldNorm(doc=2446)
          0.03828839 = weight(abstract_txt:context in 2446) [ClassicSimilarity], result of:
            0.03828839 = score(doc=2446,freq=2.0), product of:
              0.09946042 = queryWeight, product of:
                1.8715669 = boost
                4.355337 = idf(docFreq=1519, maxDocs=43556)
                0.012201779 = queryNorm
              0.38496104 = fieldWeight in 2446, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.355337 = idf(docFreq=1519, maxDocs=43556)
                0.0625 = fieldNorm(doc=2446)
          0.09135732 = weight(abstract_txt:word in 2446) [ClassicSimilarity], result of:
            0.09135732 = score(doc=2446,freq=3.0), product of:
              0.15514486 = queryWeight, product of:
                2.3374853 = boost
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.012201779 = queryNorm
              0.58885175 = fieldWeight in 2446, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.4395795 = idf(docFreq=513, maxDocs=43556)
                0.0625 = fieldNorm(doc=2446)
          0.14224409 = weight(abstract_txt:words in 2446) [ClassicSimilarity], result of:
            0.14224409 = score(doc=2446,freq=2.0), product of:
              0.30058804 = queryWeight, product of:
                4.6013055 = boost
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.012201779 = queryNorm
              0.47321936 = fieldWeight in 2446, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.353866 = idf(docFreq=559, maxDocs=43556)
                0.0625 = fieldNorm(doc=2446)
          0.29680675 = weight(abstract_txt:factor in 2446) [ClassicSimilarity], result of:
            0.29680675 = score(doc=2446,freq=3.0), product of:
              0.47193325 = queryWeight, product of:
                6.6574035 = boost
                5.8096848 = idf(docFreq=354, maxDocs=43556)
                0.012201779 = queryNorm
              0.6289168 = fieldWeight in 2446, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                5.8096848 = idf(docFreq=354, maxDocs=43556)
                0.0625 = fieldNorm(doc=2446)
        0.24 = coord(6/25)