Document (#28326)

Author
Goren-Bar, D.
Kuflik, T.
Title
Supporting user-subjective categorization with self-organizing maps and learning vector quantization
Source
Journal of the American Society for Information Science and Technology. 56(2005) no.4, S.345-355
Year
2005
Abstract
Today, most document categorization in organizations is done manually. We save at work hundreds of files and e-mail messages in folders every day. While automatic document categorization has been widely studied, much challenging research still remains to support usersubjective categorization. This study evaluates and compares the application of self-organizing maps (SOMs) and learning vector quantization (LVO) with automatic document classification, using a set of documents from an organization, in a specific domain, manually classified by a domain expert. After running the SOM and LVO we requested the user to reclassify documents that were misclassified by the system. Results show that despite the subjective nature of human categorization, automatic document categorization methods correlate weIl with subjective, personal categorization, and the LVO method outperforms the SOM. The reclassification process revealed an interesting pattern: About 40% of the documents were classified according to their original categorization, about 35% according to the system's categorization (the users changed the original categorization), and the remainder received a different (new) categorization. Based an these results we conclude that automatic support for subjective categorization is feasible; however, an exact match is probably impossible due to the users' changing categorization behavior.
Object
SOM

Similar documents (content)

  1. Roitblat, H.L.; Kershaw, A.; Oot, P.: Document categorization in legal electronic discovery : computer classification vs. manual review (2009) 0.22 
    0.22028823 = sum of:
  0.22028823 = product of:
    0.91786766 = sum of:
      0.045054484 = weight(abstract_txt:requested in 3314) [ClassicSimilarity], result of:
        0.045054484 = score(doc=3314,freq=1.0), product of:
          0.09057068 = queryWeight, product of:
            1.0326047 = boost
            7.9592175 = idf(docFreq=41, maxDocs=44218)
            0.01102004 = queryNorm
          0.4974511 = fieldWeight in 3314, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.9592175 = idf(docFreq=41, maxDocs=44218)
            0.0625 = fieldNorm(doc=3314)
      0.014999879 = weight(abstract_txt:support in 3314) [ClassicSimilarity], result of:
        0.014999879 = score(doc=3314,freq=1.0), product of:
          0.054814752 = queryWeight, product of:
            1.1360672 = boost
            4.378348 = idf(docFreq=1507, maxDocs=44218)
            0.01102004 = queryNorm
          0.27364674 = fieldWeight in 3314, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.378348 = idf(docFreq=1507, maxDocs=44218)
            0.0625 = fieldNorm(doc=3314)
      0.02816627 = weight(abstract_txt:original in 3314) [ClassicSimilarity], result of:
        0.02816627 = score(doc=3314,freq=1.0), product of:
          0.083430566 = queryWeight, product of:
            1.40158 = boost
            5.4016213 = idf(docFreq=541, maxDocs=44218)
            0.01102004 = queryNorm
          0.33760133 = fieldWeight in 3314, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.4016213 = idf(docFreq=541, maxDocs=44218)
            0.0625 = fieldNorm(doc=3314)
      0.04596501 = weight(abstract_txt:documents in 3314) [ClassicSimilarity], result of:
        0.04596501 = score(doc=3314,freq=6.0), product of:
          0.07285128 = queryWeight, product of:
            1.6040559 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.01102004 = queryNorm
          0.63094306 = fieldWeight in 3314, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=3314)
      0.03998211 = weight(abstract_txt:document in 3314) [ClassicSimilarity], result of:
        0.03998211 = score(doc=3314,freq=2.0), product of:
          0.105377875 = queryWeight, product of:
            2.22764 = boost
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.01102004 = queryNorm
          0.37941656 = fieldWeight in 3314, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.0625 = fieldNorm(doc=3314)
      0.7436999 = weight(abstract_txt:categorization in 3314) [ClassicSimilarity], result of:
        0.7436999 = score(doc=3314,freq=5.0), product of:
          0.8073935 = queryWeight, product of:
            11.116155 = boost
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.01102004 = queryNorm
          0.9211121 = fieldWeight in 3314, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.0625 = fieldNorm(doc=3314)
    0.24 = coord(6/25)
  2. Westman, S.; Laine-Hernandez, M.; Oittinen, P.: Development and evaluation of a multifaceted magazine image categorization model (2011) 0.17 
    0.16955408 = sum of:
  0.16955408 = product of:
    0.8477704 = sum of:
      0.010774846 = weight(abstract_txt:about in 4193) [ClassicSimilarity], result of:
        0.010774846 = score(doc=4193,freq=1.0), product of:
          0.0439656 = queryWeight, product of:
            1.017447 = boost
            3.9211915 = idf(docFreq=2381, maxDocs=44218)
            0.01102004 = queryNorm
          0.24507447 = fieldWeight in 4193, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.9211915 = idf(docFreq=2381, maxDocs=44218)
            0.0625 = fieldNorm(doc=4193)
      0.014999879 = weight(abstract_txt:support in 4193) [ClassicSimilarity], result of:
        0.014999879 = score(doc=4193,freq=1.0), product of:
          0.054814752 = queryWeight, product of:
            1.1360672 = boost
            4.378348 = idf(docFreq=1507, maxDocs=44218)
            0.01102004 = queryNorm
          0.27364674 = fieldWeight in 4193, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.378348 = idf(docFreq=1507, maxDocs=44218)
            0.0625 = fieldNorm(doc=4193)
      0.02816627 = weight(abstract_txt:original in 4193) [ClassicSimilarity], result of:
        0.02816627 = score(doc=4193,freq=1.0), product of:
          0.083430566 = queryWeight, product of:
            1.40158 = boost
            5.4016213 = idf(docFreq=541, maxDocs=44218)
            0.01102004 = queryNorm
          0.33760133 = fieldWeight in 4193, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.4016213 = idf(docFreq=541, maxDocs=44218)
            0.0625 = fieldNorm(doc=4193)
      0.050129507 = weight(abstract_txt:automatic in 4193) [ClassicSimilarity], result of:
        0.050129507 = score(doc=4193,freq=1.0), product of:
          0.15437533 = queryWeight, product of:
            2.6962442 = boost
            5.1955976 = idf(docFreq=665, maxDocs=44218)
            0.01102004 = queryNorm
          0.32472485 = fieldWeight in 4193, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.1955976 = idf(docFreq=665, maxDocs=44218)
            0.0625 = fieldNorm(doc=4193)
      0.7436999 = weight(abstract_txt:categorization in 4193) [ClassicSimilarity], result of:
        0.7436999 = score(doc=4193,freq=5.0), product of:
          0.8073935 = queryWeight, product of:
            11.116155 = boost
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.01102004 = queryNorm
          0.9211121 = fieldWeight in 4193, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.0625 = fieldNorm(doc=4193)
    0.2 = coord(5/25)
  3. Sah, M.; Wade, V.: Personalized concept-based search on the Linked Open Data (2015) 0.16 
    0.16279072 = sum of:
  0.16279072 = product of:
    0.8139536 = sum of:
      0.018561402 = weight(abstract_txt:support in 2511) [ClassicSimilarity], result of:
        0.018561402 = score(doc=2511,freq=2.0), product of:
          0.054814752 = queryWeight, product of:
            1.1360672 = boost
            4.378348 = idf(docFreq=1507, maxDocs=44218)
            0.01102004 = queryNorm
          0.33862057 = fieldWeight in 2511, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.378348 = idf(docFreq=1507, maxDocs=44218)
            0.0546875 = fieldNorm(doc=2511)
      0.016606865 = weight(abstract_txt:domain in 2511) [ClassicSimilarity], result of:
        0.016606865 = score(doc=2511,freq=1.0), product of:
          0.0641247 = queryWeight, product of:
            1.2287625 = boost
            4.7355914 = idf(docFreq=1054, maxDocs=44218)
            0.01102004 = queryNorm
          0.25897765 = fieldWeight in 2511, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.7355914 = idf(docFreq=1054, maxDocs=44218)
            0.0546875 = fieldNorm(doc=2511)
      0.022581525 = weight(abstract_txt:according in 2511) [ClassicSimilarity], result of:
        0.022581525 = score(doc=2511,freq=1.0), product of:
          0.07870502 = queryWeight, product of:
            1.3613083 = boost
            5.2464166 = idf(docFreq=632, maxDocs=44218)
            0.01102004 = queryNorm
          0.2869134 = fieldWeight in 2511, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.2464166 = idf(docFreq=632, maxDocs=44218)
            0.0546875 = fieldNorm(doc=2511)
      0.043356597 = weight(abstract_txt:vector in 2511) [ClassicSimilarity], result of:
        0.043356597 = score(doc=2511,freq=1.0), product of:
          0.12158231 = queryWeight, product of:
            1.6919614 = boost
            6.5207376 = idf(docFreq=176, maxDocs=44218)
            0.01102004 = queryNorm
          0.35660285 = fieldWeight in 2511, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.5207376 = idf(docFreq=176, maxDocs=44218)
            0.0546875 = fieldNorm(doc=2511)
      0.7128472 = weight(abstract_txt:categorization in 2511) [ClassicSimilarity], result of:
        0.7128472 = score(doc=2511,freq=6.0), product of:
          0.8073935 = queryWeight, product of:
            11.116155 = boost
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.01102004 = queryNorm
          0.88289934 = fieldWeight in 2511, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.0546875 = fieldNorm(doc=2511)
    0.2 = coord(5/25)
  4. Kim, J.-H.; Choi, K.-S.: Patent document categorization based on semantic structural information (2007) 0.16 
    0.16256545 = sum of:
  0.16256545 = product of:
    1.0160341 = sum of:
      0.04596501 = weight(abstract_txt:documents in 933) [ClassicSimilarity], result of:
        0.04596501 = score(doc=933,freq=6.0), product of:
          0.07285128 = queryWeight, product of:
            1.6040559 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.01102004 = queryNorm
          0.63094306 = fieldWeight in 933, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=933)
      0.03998211 = weight(abstract_txt:document in 933) [ClassicSimilarity], result of:
        0.03998211 = score(doc=933,freq=2.0), product of:
          0.105377875 = queryWeight, product of:
            2.22764 = boost
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.01102004 = queryNorm
          0.37941656 = fieldWeight in 933, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.0625 = fieldNorm(doc=933)
      0.050129507 = weight(abstract_txt:automatic in 933) [ClassicSimilarity], result of:
        0.050129507 = score(doc=933,freq=1.0), product of:
          0.15437533 = queryWeight, product of:
            2.6962442 = boost
            5.1955976 = idf(docFreq=665, maxDocs=44218)
            0.01102004 = queryNorm
          0.32472485 = fieldWeight in 933, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.1955976 = idf(docFreq=665, maxDocs=44218)
            0.0625 = fieldNorm(doc=933)
      0.87995756 = weight(abstract_txt:categorization in 933) [ClassicSimilarity], result of:
        0.87995756 = score(doc=933,freq=7.0), product of:
          0.8073935 = queryWeight, product of:
            11.116155 = boost
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.01102004 = queryNorm
          1.0898745 = fieldWeight in 933, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.0625 = fieldNorm(doc=933)
    0.16 = coord(4/25)
  5. Humphrey, S.M.; Névéol, A.; Browne, A.; Gobeil, J.; Ruch, P.; Darmoni, S.J.: Comparing a rule-based versus statistical system for automatic categorization of MEDLINE documents according to biomedical specialty (2009) 0.16 
    0.16253121 = sum of:
  0.16253121 = product of:
    0.81265604 = sum of:
      0.010774846 = weight(abstract_txt:about in 3300) [ClassicSimilarity], result of:
        0.010774846 = score(doc=3300,freq=1.0), product of:
          0.0439656 = queryWeight, product of:
            1.017447 = boost
            3.9211915 = idf(docFreq=2381, maxDocs=44218)
            0.01102004 = queryNorm
          0.24507447 = fieldWeight in 3300, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.9211915 = idf(docFreq=2381, maxDocs=44218)
            0.0625 = fieldNorm(doc=3300)
      0.03753027 = weight(abstract_txt:documents in 3300) [ClassicSimilarity], result of:
        0.03753027 = score(doc=3300,freq=4.0), product of:
          0.07285128 = queryWeight, product of:
            1.6040559 = boost
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.01102004 = queryNorm
          0.5151628 = fieldWeight in 3300, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.1213026 = idf(docFreq=1949, maxDocs=44218)
            0.0625 = fieldNorm(doc=3300)
      0.028271621 = weight(abstract_txt:document in 3300) [ClassicSimilarity], result of:
        0.028271621 = score(doc=3300,freq=1.0), product of:
          0.105377875 = queryWeight, product of:
            2.22764 = boost
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.01102004 = queryNorm
          0.26828802 = fieldWeight in 3300, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.2926083 = idf(docFreq=1642, maxDocs=44218)
            0.0625 = fieldNorm(doc=3300)
      0.070893824 = weight(abstract_txt:automatic in 3300) [ClassicSimilarity], result of:
        0.070893824 = score(doc=3300,freq=2.0), product of:
          0.15437533 = queryWeight, product of:
            2.6962442 = boost
            5.1955976 = idf(docFreq=665, maxDocs=44218)
            0.01102004 = queryNorm
          0.45923027 = fieldWeight in 3300, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.1955976 = idf(docFreq=665, maxDocs=44218)
            0.0625 = fieldNorm(doc=3300)
      0.66518545 = weight(abstract_txt:categorization in 3300) [ClassicSimilarity], result of:
        0.66518545 = score(doc=3300,freq=4.0), product of:
          0.8073935 = queryWeight, product of:
            11.116155 = boost
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.01102004 = queryNorm
          0.82386774 = fieldWeight in 3300, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            6.590942 = idf(docFreq=164, maxDocs=44218)
            0.0625 = fieldNorm(doc=3300)
    0.2 = coord(5/25)