Document (#43161)

Author
Lee, G.E.
Sun, A.
Title
Understanding the stability of medical concept embeddings
Source
Journal of the Association for Information Science and Technology. 72(2021) no.3, S.346-356
Year
2021
Abstract
Frequency is one of the major factors for training quality word embeddings. Several studies have recently discussed the stability of word embeddings in general domain and suggested factors influencing the stability. In this work, we conduct a detailed analysis on the stability of concept embeddings in medical domain, particularly in relations with concept frequency. The analysis reveals the surprising high stability of low-frequency concepts: low-frequency (<100) concepts have the same high stability as high-frequency (>1,000) concepts. To develop a deeper understanding of this finding, we propose a new factor, the noisiness of context words, which influences the stability of medical concept embeddings regardless of high or low frequency. We evaluate the proposed factor by showing the linear correlation with the stability of medical concept embeddings. The correlations are clear and consistent with various groups of medical concepts. Based on the linear relations, we make suggestions on ways to adjust the noisiness of context words for the improvement of stability. Finally, we demonstrate that the linear relation of the proposed factor extends to the word embedding stability in general domain.
Content
Vgl.: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.24411.
Theme
Computerlinguistik
Field
Medizin

Similar documents (content)

  1. Li, G.; Siddharth, L.; Luo, J.: Embedding knowledge graph of patent metadata to measure knowledge proximity (2023) 0.18 
    0.18031172 = sum of:
  0.18031172 = product of:
    0.90155864 = sum of:
      0.060070477 = weight(abstract_txt:embedding in 920) [ClassicSimilarity], result of:
        0.060070477 = score(doc=920,freq=2.0), product of:
          0.0692894 = queryWeight, product of:
            1.0738529 = boost
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.0082230475 = queryNorm
          0.86695045 = fieldWeight in 920, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.078125 = fieldNorm(doc=920)
      0.01437369 = weight(abstract_txt:context in 920) [ClassicSimilarity], result of:
        0.01437369 = score(doc=920,freq=1.0), product of:
          0.04239275 = queryWeight, product of:
            1.1878788 = boost
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0082230475 = queryNorm
          0.3390601 = fieldWeight in 920, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.078125 = fieldNorm(doc=920)
      0.029999552 = weight(abstract_txt:relations in 920) [ClassicSimilarity], result of:
        0.029999552 = score(doc=920,freq=1.0), product of:
          0.069234505 = queryWeight, product of:
            1.5180556 = boost
            5.5462847 = idf(docFreq=468, maxDocs=44218)
            0.0082230475 = queryNorm
          0.43330348 = fieldWeight in 920, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.5462847 = idf(docFreq=468, maxDocs=44218)
            0.078125 = fieldNorm(doc=920)
      0.039612953 = weight(abstract_txt:domain in 920) [ClassicSimilarity], result of:
        0.039612953 = score(doc=920,freq=2.0), product of:
          0.075710826 = queryWeight, product of:
            1.944245 = boost
            4.7355914 = idf(docFreq=1054, maxDocs=44218)
            0.0082230475 = queryNorm
          0.52321386 = fieldWeight in 920, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.7355914 = idf(docFreq=1054, maxDocs=44218)
            0.078125 = fieldNorm(doc=920)
      0.75750196 = weight(abstract_txt:embeddings in 920) [ClassicSimilarity], result of:
        0.75750196 = score(doc=920,freq=3.0), product of:
          0.59589344 = queryWeight, product of:
            7.713852 = boost
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.0082230475 = queryNorm
          1.2712038 = fieldWeight in 920, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.078125 = fieldNorm(doc=920)
    0.2 = coord(5/25)
  2. Lee, Y.-Y.; Ke, H.; Yen, T.-Y.; Huang, H.-H.; Chen, H.-H.: Combining and learning word embedding with WordNet for semantic relatedness and similarity measurement (2020) 0.15 
    0.15297303 = sum of:
  0.15297303 = product of:
    0.63738763 = sum of:
      0.03552296 = weight(abstract_txt:correlations in 5871) [ClassicSimilarity], result of:
        0.03552296 = score(doc=5871,freq=1.0), product of:
          0.061504778 = queryWeight, product of:
            1.0117327 = boost
            7.3928223 = idf(docFreq=73, maxDocs=44218)
            0.0082230475 = queryNorm
          0.57756424 = fieldWeight in 5871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.3928223 = idf(docFreq=73, maxDocs=44218)
            0.078125 = fieldNorm(doc=5871)
      0.0041198432 = weight(abstract_txt:with in 5871) [ClassicSimilarity], result of:
        0.0041198432 = score(doc=5871,freq=1.0), product of:
          0.021095848 = queryWeight, product of:
            1.0262909 = boost
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0082230475 = queryNorm
          0.19529167 = fieldWeight in 5871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.078125 = fieldNorm(doc=5871)
      0.060070477 = weight(abstract_txt:embedding in 5871) [ClassicSimilarity], result of:
        0.060070477 = score(doc=5871,freq=2.0), product of:
          0.0692894 = queryWeight, product of:
            1.0738529 = boost
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.0082230475 = queryNorm
          0.86695045 = fieldWeight in 5871, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.078125 = fieldNorm(doc=5871)
      0.026971292 = weight(abstract_txt:words in 5871) [ClassicSimilarity], result of:
        0.026971292 = score(doc=5871,freq=1.0), product of:
          0.0644932 = queryWeight, product of:
            1.4651542 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0082230475 = queryNorm
          0.41820365 = fieldWeight in 5871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.078125 = fieldNorm(doc=5871)
      0.07335905 = weight(abstract_txt:word in 5871) [ClassicSimilarity], result of:
        0.07335905 = score(doc=5871,freq=3.0), product of:
          0.09974058 = queryWeight, product of:
            2.2315567 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0082230475 = queryNorm
          0.73549855 = fieldWeight in 5871, product of:
            1.7320508 = tf(freq=3.0), with freq of:
              3.0 = termFreq=3.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.078125 = fieldNorm(doc=5871)
      0.43734398 = weight(abstract_txt:embeddings in 5871) [ClassicSimilarity], result of:
        0.43734398 = score(doc=5871,freq=1.0), product of:
          0.59589344 = queryWeight, product of:
            7.713852 = boost
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.0082230475 = queryNorm
          0.7339299 = fieldWeight in 5871, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.078125 = fieldNorm(doc=5871)
    0.24 = coord(6/25)
  3. Safder, I.; Ali, M.; Aljohani, N.R.; Nawaz, R.; Hassan, S.-U.: Neural machine translation for in-text citation classification (2023) 0.15 
    0.15063784 = sum of:
  0.15063784 = product of:
    0.62765765 = sum of:
      0.0046610706 = weight(abstract_txt:with in 1053) [ClassicSimilarity], result of:
        0.0046610706 = score(doc=1053,freq=2.0), product of:
          0.021095848 = queryWeight, product of:
            1.0262909 = boost
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0082230475 = queryNorm
          0.22094731 = fieldWeight in 1053, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0625 = fieldNorm(doc=1053)
      0.033980995 = weight(abstract_txt:embedding in 1053) [ClassicSimilarity], result of:
        0.033980995 = score(doc=1053,freq=1.0), product of:
          0.0692894 = queryWeight, product of:
            1.0738529 = boost
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.0082230475 = queryNorm
          0.49042124 = fieldWeight in 1053, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.0625 = fieldNorm(doc=1053)
      0.03252395 = weight(abstract_txt:context in 1053) [ClassicSimilarity], result of:
        0.03252395 = score(doc=1053,freq=8.0), product of:
          0.04239275 = queryWeight, product of:
            1.1878788 = boost
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0082230475 = queryNorm
          0.7672054 = fieldWeight in 1053, product of:
            2.828427 = tf(freq=8.0), with freq of:
              8.0 = termFreq=8.0
            4.339969 = idf(docFreq=1566, maxDocs=44218)
            0.0625 = fieldNorm(doc=1053)
      0.01377549 = weight(abstract_txt:proposed in 1053) [ClassicSimilarity], result of:
        0.01377549 = score(doc=1053,freq=1.0), product of:
          0.047817934 = queryWeight, product of:
            1.2616003 = boost
            4.6093135 = idf(docFreq=1196, maxDocs=44218)
            0.0082230475 = queryNorm
          0.2880821 = fieldWeight in 1053, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.6093135 = idf(docFreq=1196, maxDocs=44218)
            0.0625 = fieldNorm(doc=1053)
      0.047917932 = weight(abstract_txt:word in 1053) [ClassicSimilarity], result of:
        0.047917932 = score(doc=1053,freq=2.0), product of:
          0.09974058 = queryWeight, product of:
            2.2315567 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0082230475 = queryNorm
          0.48042563 = fieldWeight in 1053, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0625 = fieldNorm(doc=1053)
      0.4947982 = weight(abstract_txt:embeddings in 1053) [ClassicSimilarity], result of:
        0.4947982 = score(doc=1053,freq=2.0), product of:
          0.59589344 = queryWeight, product of:
            7.713852 = boost
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.0082230475 = queryNorm
          0.8303468 = fieldWeight in 1053, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.0625 = fieldNorm(doc=1053)
    0.24 = coord(6/25)
  4. Sun, Q.; Shaw, D.; Davis, C.H.: ¬A model for estimating the occurence of same-frequency words and the boundary between high- and low-frequency words in texts (1999) 0.14 
    0.13836965 = sum of:
  0.13836965 = product of:
    0.57654023 = sum of:
      0.0049438123 = weight(abstract_txt:with in 3063) [ClassicSimilarity], result of:
        0.0049438123 = score(doc=3063,freq=1.0), product of:
          0.021095848 = queryWeight, product of:
            1.0262909 = boost
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0082230475 = queryNorm
          0.23435001 = fieldWeight in 3063, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.09375 = fieldNorm(doc=3063)
      0.020663235 = weight(abstract_txt:proposed in 3063) [ClassicSimilarity], result of:
        0.020663235 = score(doc=3063,freq=1.0), product of:
          0.047817934 = queryWeight, product of:
            1.2616003 = boost
            4.6093135 = idf(docFreq=1196, maxDocs=44218)
            0.0082230475 = queryNorm
          0.43212312 = fieldWeight in 3063, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.6093135 = idf(docFreq=1196, maxDocs=44218)
            0.09375 = fieldNorm(doc=3063)
      0.07927909 = weight(abstract_txt:words in 3063) [ClassicSimilarity], result of:
        0.07927909 = score(doc=3063,freq=6.0), product of:
          0.0644932 = queryWeight, product of:
            1.4651542 = boost
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.0082230475 = queryNorm
          1.2292627 = fieldWeight in 3063, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            5.353007 = idf(docFreq=568, maxDocs=44218)
            0.09375 = fieldNorm(doc=3063)
      0.050824642 = weight(abstract_txt:word in 3063) [ClassicSimilarity], result of:
        0.050824642 = score(doc=3063,freq=1.0), product of:
          0.09974058 = queryWeight, product of:
            2.2315567 = boost
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.0082230475 = queryNorm
          0.50956833 = fieldWeight in 3063, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.4353957 = idf(docFreq=523, maxDocs=44218)
            0.09375 = fieldNorm(doc=3063)
      0.06848949 = weight(abstract_txt:high in 3063) [ClassicSimilarity], result of:
        0.06848949 = score(doc=3063,freq=2.0), product of:
          0.10630195 = queryWeight, product of:
            2.6601858 = boost
            4.8595543 = idf(docFreq=931, maxDocs=44218)
            0.0082230475 = queryNorm
          0.64429194 = fieldWeight in 3063, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.8595543 = idf(docFreq=931, maxDocs=44218)
            0.09375 = fieldNorm(doc=3063)
      0.35233998 = weight(abstract_txt:frequency in 3063) [ClassicSimilarity], result of:
        0.35233998 = score(doc=3063,freq=7.0), product of:
          0.2388403 = queryWeight, product of:
            4.883608 = boost
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.0082230475 = queryNorm
          1.4752116 = fieldWeight in 3063, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            5.947494 = idf(docFreq=313, maxDocs=44218)
            0.09375 = fieldNorm(doc=3063)
    0.24 = coord(6/25)
  5. Maaten, L. van den: Accelerating t-SNE using Tree-Based Algorithms (2014) 0.14 
    0.13544671 = sum of:
  0.13544671 = product of:
    0.846542 = sum of:
      0.0049438123 = weight(abstract_txt:with in 3886) [ClassicSimilarity], result of:
        0.0049438123 = score(doc=3886,freq=1.0), product of:
          0.021095848 = queryWeight, product of:
            1.0262909 = boost
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.0082230475 = queryNorm
          0.23435001 = fieldWeight in 3886, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            2.4997334 = idf(docFreq=9868, maxDocs=44218)
            0.09375 = fieldNorm(doc=3886)
      0.05097149 = weight(abstract_txt:embedding in 3886) [ClassicSimilarity], result of:
        0.05097149 = score(doc=3886,freq=1.0), product of:
          0.0692894 = queryWeight, product of:
            1.0738529 = boost
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.0082230475 = queryNorm
          0.7356318 = fieldWeight in 3886, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.84674 = idf(docFreq=46, maxDocs=44218)
            0.09375 = fieldNorm(doc=3886)
      0.048429385 = weight(abstract_txt:high in 3886) [ClassicSimilarity], result of:
        0.048429385 = score(doc=3886,freq=1.0), product of:
          0.10630195 = queryWeight, product of:
            2.6601858 = boost
            4.8595543 = idf(docFreq=931, maxDocs=44218)
            0.0082230475 = queryNorm
          0.4555832 = fieldWeight in 3886, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.8595543 = idf(docFreq=931, maxDocs=44218)
            0.09375 = fieldNorm(doc=3886)
      0.74219733 = weight(abstract_txt:embeddings in 3886) [ClassicSimilarity], result of:
        0.74219733 = score(doc=3886,freq=2.0), product of:
          0.59589344 = queryWeight, product of:
            7.713852 = boost
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.0082230475 = queryNorm
          1.2455202 = fieldWeight in 3886, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            9.394302 = idf(docFreq=9, maxDocs=44218)
            0.09375 = fieldNorm(doc=3886)
    0.16 = coord(4/25)