Document (#28324)

Author
Goren-Bar, D.
Kuflik, T.
Title
Supporting user-subjective categorization with self-organizing maps and learning vector quantization
Source
Journal of the American Society for Information Science and Technology. 56(2005) no.4, S.345-355
Year
2005
Abstract
Today, most document categorization in organizations is done manually. We save at work hundreds of files and e-mail messages in folders every day. While automatic document categorization has been widely studied, much challenging research still remains to support usersubjective categorization. This study evaluates and compares the application of self-organizing maps (SOMs) and learning vector quantization (LVO) with automatic document classification, using a set of documents from an organization, in a specific domain, manually classified by a domain expert. After running the SOM and LVO we requested the user to reclassify documents that were misclassified by the system. Results show that despite the subjective nature of human categorization, automatic document categorization methods correlate weIl with subjective, personal categorization, and the LVO method outperforms the SOM. The reclassification process revealed an interesting pattern: About 40% of the documents were classified according to their original categorization, about 35% according to the system's categorization (the users changed the original categorization), and the remainder received a different (new) categorization. Based an these results we conclude that automatic support for subjective categorization is feasible; however, an exact match is probably impossible due to the users' changing categorization behavior.
Object
SOM

Similar documents (content)

  1. Roitblat, H.L.; Kershaw, A.; Oot, P.: Document categorization in legal electronic discovery : computer classification vs. manual review (2009) 0.22 
    0.22146174 = sum of:
  0.22146174 = product of:
    0.92275727 = sum of:
      0.044473954 = weight(abstract_txt:requested in 312) [ClassicSimilarity], result of:
        0.044473954 = score(doc=312,freq=1.0), product of:
          0.08957343 = queryWeight, product of:
            1.0301657 = boost
            7.9441333 = idf(docFreq=41, maxDocs=43556)
            0.010945248 = queryNorm
          0.49650833 = fieldWeight in 312, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            7.9441333 = idf(docFreq=41, maxDocs=43556)
            0.0625 = fieldNorm(doc=312)
      0.015074986 = weight(abstract_txt:support in 312) [ClassicSimilarity], result of:
        0.015074986 = score(doc=312,freq=1.0), product of:
          0.054864317 = queryWeight, product of:
            1.1401906 = boost
            4.396296 = idf(docFreq=1458, maxDocs=43556)
            0.010945248 = queryNorm
          0.2747685 = fieldWeight in 312, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.396296 = idf(docFreq=1458, maxDocs=43556)
            0.0625 = fieldNorm(doc=312)
      0.028017003 = weight(abstract_txt:original in 312) [ClassicSimilarity], result of:
        0.028017003 = score(doc=312,freq=1.0), product of:
          0.082934104 = queryWeight, product of:
            1.4018419 = boost
            5.4051595 = idf(docFreq=531, maxDocs=43556)
            0.010945248 = queryNorm
          0.33782247 = fieldWeight in 312, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.4051595 = idf(docFreq=531, maxDocs=43556)
            0.0625 = fieldNorm(doc=312)
      0.045611057 = weight(abstract_txt:documents in 312) [ClassicSimilarity], result of:
        0.045611057 = score(doc=312,freq=6.0), product of:
          0.072301194 = queryWeight, product of:
            1.6030638 = boost
            4.1206813 = idf(docFreq=1921, maxDocs=43556)
            0.010945248 = queryNorm
          0.63084793 = fieldWeight in 312, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            4.1206813 = idf(docFreq=1921, maxDocs=43556)
            0.0625 = fieldNorm(doc=312)
      0.03957943 = weight(abstract_txt:document in 312) [ClassicSimilarity], result of:
        0.03957943 = score(doc=312,freq=2.0), product of:
          0.10441552 = queryWeight, product of:
            2.2244875 = boost
            4.28854 = idf(docFreq=1624, maxDocs=43556)
            0.010945248 = queryNorm
          0.37905696 = fieldWeight in 312, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.28854 = idf(docFreq=1624, maxDocs=43556)
            0.0625 = fieldNorm(doc=312)
      0.75000083 = weight(abstract_txt:categorization in 312) [ClassicSimilarity], result of:
        0.75000083 = score(doc=312,freq=5.0), product of:
          0.80998003 = queryWeight, product of:
            11.169309 = boost
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.010945248 = queryNorm
          0.92594975 = fieldWeight in 312, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.0625 = fieldNorm(doc=312)
    0.24 = coord(6/25)
  2. Westman, S.; Laine-Hernandez, M.; Oittinen, P.: Development and evaluation of a multifaceted magazine image categorization model (2011) 0.17 
    0.17072989 = sum of:
  0.17072989 = product of:
    0.85364944 = sum of:
      0.010789364 = weight(abstract_txt:about in 1191) [ClassicSimilarity], result of:
        0.010789364 = score(doc=1191,freq=1.0), product of:
          0.043898456 = queryWeight, product of:
            1.0198988 = boost
            3.9324803 = idf(docFreq=2319, maxDocs=43556)
            0.010945248 = queryNorm
          0.24578002 = fieldWeight in 1191, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.9324803 = idf(docFreq=2319, maxDocs=43556)
            0.0625 = fieldNorm(doc=1191)
      0.015074986 = weight(abstract_txt:support in 1191) [ClassicSimilarity], result of:
        0.015074986 = score(doc=1191,freq=1.0), product of:
          0.054864317 = queryWeight, product of:
            1.1401906 = boost
            4.396296 = idf(docFreq=1458, maxDocs=43556)
            0.010945248 = queryNorm
          0.2747685 = fieldWeight in 1191, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.396296 = idf(docFreq=1458, maxDocs=43556)
            0.0625 = fieldNorm(doc=1191)
      0.028017003 = weight(abstract_txt:original in 1191) [ClassicSimilarity], result of:
        0.028017003 = score(doc=1191,freq=1.0), product of:
          0.082934104 = queryWeight, product of:
            1.4018419 = boost
            5.4051595 = idf(docFreq=531, maxDocs=43556)
            0.010945248 = queryNorm
          0.33782247 = fieldWeight in 1191, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.4051595 = idf(docFreq=531, maxDocs=43556)
            0.0625 = fieldNorm(doc=1191)
      0.04976727 = weight(abstract_txt:automatic in 1191) [ClassicSimilarity], result of:
        0.04976727 = score(doc=1191,freq=1.0), product of:
          0.1532585 = queryWeight, product of:
            2.6950061 = boost
            5.195642 = idf(docFreq=655, maxDocs=43556)
            0.010945248 = queryNorm
          0.32472762 = fieldWeight in 1191, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.195642 = idf(docFreq=655, maxDocs=43556)
            0.0625 = fieldNorm(doc=1191)
      0.75000083 = weight(abstract_txt:categorization in 1191) [ClassicSimilarity], result of:
        0.75000083 = score(doc=1191,freq=5.0), product of:
          0.80998003 = queryWeight, product of:
            11.169309 = boost
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.010945248 = queryNorm
          0.92594975 = fieldWeight in 1191, product of:
            2.236068 = tf(freq=5.0), with freq of:
              5.0 = termFreq=5.0
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.0625 = fieldNorm(doc=1191)
    0.2 = coord(5/25)
  3. Sah, M.; Wade, V.: Personalized concept-based search on the Linked Open Data (2015) 0.16 
    0.16395509 = sum of:
  0.16395509 = product of:
    0.81977546 = sum of:
      0.018654345 = weight(abstract_txt:support in 4509) [ClassicSimilarity], result of:
        0.018654345 = score(doc=4509,freq=2.0), product of:
          0.054864317 = queryWeight, product of:
            1.1401906 = boost
            4.396296 = idf(docFreq=1458, maxDocs=43556)
            0.010945248 = queryNorm
          0.34000868 = fieldWeight in 4509, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.396296 = idf(docFreq=1458, maxDocs=43556)
            0.0546875 = fieldNorm(doc=4509)
      0.016713051 = weight(abstract_txt:domain in 4509) [ClassicSimilarity], result of:
        0.016713051 = score(doc=4509,freq=1.0), product of:
          0.06424172 = queryWeight, product of:
            1.2337894 = boost
            4.75719 = idf(docFreq=1016, maxDocs=43556)
            0.010945248 = queryNorm
          0.26015884 = fieldWeight in 4509, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.75719 = idf(docFreq=1016, maxDocs=43556)
            0.0546875 = fieldNorm(doc=4509)
      0.022552863 = weight(abstract_txt:according in 4509) [ClassicSimilarity], result of:
        0.022552863 = score(doc=4509,freq=1.0), product of:
          0.07844786 = queryWeight, product of:
            1.3633991 = boost
            5.2569337 = idf(docFreq=616, maxDocs=43556)
            0.010945248 = queryNorm
          0.28748855 = fieldWeight in 4509, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.2569337 = idf(docFreq=616, maxDocs=43556)
            0.0546875 = fieldNorm(doc=4509)
      0.042968556 = weight(abstract_txt:vector in 4509) [ClassicSimilarity], result of:
        0.042968556 = score(doc=4509,freq=1.0), product of:
          0.12056295 = queryWeight, product of:
            1.690205 = boost
            6.517017 = idf(docFreq=174, maxDocs=43556)
            0.010945248 = queryNorm
          0.35639936 = fieldWeight in 4509, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            6.517017 = idf(docFreq=174, maxDocs=43556)
            0.0546875 = fieldNorm(doc=4509)
      0.7188867 = weight(abstract_txt:categorization in 4509) [ClassicSimilarity], result of:
        0.7188867 = score(doc=4509,freq=6.0), product of:
          0.80998003 = queryWeight, product of:
            11.169309 = boost
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.010945248 = queryNorm
          0.8875363 = fieldWeight in 4509, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.0546875 = fieldNorm(doc=4509)
    0.2 = coord(5/25)
  4. Kim, J.-H.; Choi, K.-S.: Patent document categorization based on semantic structural information (2007) 0.16 
    0.16357931 = sum of:
  0.16357931 = product of:
    1.0223707 = sum of:
      0.045611057 = weight(abstract_txt:documents in 2931) [ClassicSimilarity], result of:
        0.045611057 = score(doc=2931,freq=6.0), product of:
          0.072301194 = queryWeight, product of:
            1.6030638 = boost
            4.1206813 = idf(docFreq=1921, maxDocs=43556)
            0.010945248 = queryNorm
          0.63084793 = fieldWeight in 2931, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            4.1206813 = idf(docFreq=1921, maxDocs=43556)
            0.0625 = fieldNorm(doc=2931)
      0.03957943 = weight(abstract_txt:document in 2931) [ClassicSimilarity], result of:
        0.03957943 = score(doc=2931,freq=2.0), product of:
          0.10441552 = queryWeight, product of:
            2.2244875 = boost
            4.28854 = idf(docFreq=1624, maxDocs=43556)
            0.010945248 = queryNorm
          0.37905696 = fieldWeight in 2931, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            4.28854 = idf(docFreq=1624, maxDocs=43556)
            0.0625 = fieldNorm(doc=2931)
      0.04976727 = weight(abstract_txt:automatic in 2931) [ClassicSimilarity], result of:
        0.04976727 = score(doc=2931,freq=1.0), product of:
          0.1532585 = queryWeight, product of:
            2.6950061 = boost
            5.195642 = idf(docFreq=655, maxDocs=43556)
            0.010945248 = queryNorm
          0.32472762 = fieldWeight in 2931, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            5.195642 = idf(docFreq=655, maxDocs=43556)
            0.0625 = fieldNorm(doc=2931)
      0.8874129 = weight(abstract_txt:categorization in 2931) [ClassicSimilarity], result of:
        0.8874129 = score(doc=2931,freq=7.0), product of:
          0.80998003 = queryWeight, product of:
            11.169309 = boost
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.010945248 = queryNorm
          1.0955985 = fieldWeight in 2931, product of:
            2.6457512 = tf(freq=7.0), with freq of:
              7.0 = termFreq=7.0
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.0625 = fieldNorm(doc=2931)
    0.16 = coord(4/25)
  5. Humphrey, S.M.; Névéol, A.; Browne, A.; Gobeil, J.; Ruch, P.; Darmoni, S.J.: Comparing a rule-based versus statistical system for automatic categorization of MEDLINE documents according to biomedical specialty (2009) 0.16 
    0.16344404 = sum of:
  0.16344404 = product of:
    0.8172202 = sum of:
      0.010789364 = weight(abstract_txt:about in 298) [ClassicSimilarity], result of:
        0.010789364 = score(doc=298,freq=1.0), product of:
          0.043898456 = queryWeight, product of:
            1.0198988 = boost
            3.9324803 = idf(docFreq=2319, maxDocs=43556)
            0.010945248 = queryNorm
          0.24578002 = fieldWeight in 298, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            3.9324803 = idf(docFreq=2319, maxDocs=43556)
            0.0625 = fieldNorm(doc=298)
      0.037241273 = weight(abstract_txt:documents in 298) [ClassicSimilarity], result of:
        0.037241273 = score(doc=298,freq=4.0), product of:
          0.072301194 = queryWeight, product of:
            1.6030638 = boost
            4.1206813 = idf(docFreq=1921, maxDocs=43556)
            0.010945248 = queryNorm
          0.51508516 = fieldWeight in 298, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            4.1206813 = idf(docFreq=1921, maxDocs=43556)
            0.0625 = fieldNorm(doc=298)
      0.027986882 = weight(abstract_txt:document in 298) [ClassicSimilarity], result of:
        0.027986882 = score(doc=298,freq=1.0), product of:
          0.10441552 = queryWeight, product of:
            2.2244875 = boost
            4.28854 = idf(docFreq=1624, maxDocs=43556)
            0.010945248 = queryNorm
          0.26803374 = fieldWeight in 298, product of:
            1.0 = tf(freq=1.0), with freq of:
              1.0 = termFreq=1.0
            4.28854 = idf(docFreq=1624, maxDocs=43556)
            0.0625 = fieldNorm(doc=298)
      0.070381545 = weight(abstract_txt:automatic in 298) [ClassicSimilarity], result of:
        0.070381545 = score(doc=298,freq=2.0), product of:
          0.1532585 = queryWeight, product of:
            2.6950061 = boost
            5.195642 = idf(docFreq=655, maxDocs=43556)
            0.010945248 = queryNorm
          0.4592342 = fieldWeight in 298, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            5.195642 = idf(docFreq=655, maxDocs=43556)
            0.0625 = fieldNorm(doc=298)
      0.67082113 = weight(abstract_txt:categorization in 298) [ClassicSimilarity], result of:
        0.67082113 = score(doc=298,freq=4.0), product of:
          0.80998003 = queryWeight, product of:
            11.169309 = boost
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.010945248 = queryNorm
          0.8281946 = fieldWeight in 298, product of:
            2.0 = tf(freq=4.0), with freq of:
              4.0 = termFreq=4.0
            6.625557 = idf(docFreq=156, maxDocs=43556)
            0.0625 = fieldNorm(doc=298)
    0.2 = coord(5/25)