Document (#37757)

Author

Justeson, J.S.
Katz, S.M.

Title

Technical terminology : some linguistic properties and an algorithm for identification in text

Source

Natural language engineering. 1(1995) no.1, S.9-27

Year

1995

Abstract

This paper identifies some linguistic properties of technical terminology, and uses them to formulate an algorithm for identifying technical terms in running text. The grammatical properties discussed are preferred phrase structures: technical terms consist mostly of noun phrases containing adjectives, nouns, and occasionally prepositions; rerely do terms contain verbs, adverbs, or conjunctions. The discourse properties are patterns of repetition that distinguish noun phrases that are technical terms, especially those multi-word phrases that constitute a substantial majority of all technical vocabulary, from other types of noun phrase. The paper presents a terminology indentification algorithm that is motivated by these linguistic properties. An implementation of the algorithm is described; it recovers a high proportion of the technical terms in a text, and a high proportaion of the recovered strings are vaild technical terms. The algorithm proves to be effective regardless of the domain of the text to which it is applied.

Theme

Computerlinguistik

Similar documents (author)

1. Katz, M.: Multimedia: the future of information delivery to homes and business (1993) 5.51

   5.512169 = sum of:
     5.512169 = weight(author_txt:katz in 6646) [ClassicSimilarity], result of:
       5.512169 = fieldWeight in 6646, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.81947 = idf(docFreq=16, maxDocs=42306)
         0.625 = fieldNorm(doc=6646)
   

2. Katz, B.: Community college reference services : a working guide for and by librarians (1992) 5.51

   5.512169 = sum of:
     5.512169 = weight(author_txt:katz in 730) [ClassicSimilarity], result of:
       5.512169 = fieldWeight in 730, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.81947 = idf(docFreq=16, maxDocs=42306)
         0.625 = fieldNorm(doc=730)
   

3. Katz, J.S.: Bibliometric standards : personal experience and lessons learned (1996) 5.51

   5.512169 = sum of:
     5.512169 = weight(author_txt:katz in 5127) [ClassicSimilarity], result of:
       5.512169 = fieldWeight in 5127, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.81947 = idf(docFreq=16, maxDocs=42306)
         0.625 = fieldNorm(doc=5127)
   

4. Katz, W.A.: Introduction to reference work (1997) 5.51

   5.512169 = sum of:
     5.512169 = weight(author_txt:katz in 2189) [ClassicSimilarity], result of:
       5.512169 = fieldWeight in 2189, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.81947 = idf(docFreq=16, maxDocs=42306)
         0.625 = fieldNorm(doc=2189)
   

5. Katz, W.A.: Introduction to reference work : Vol.1: Basic information sources; vol.2: Reference services and reference processes (1992) 5.51

   5.512169 = sum of:
     5.512169 = weight(author_txt:katz in 4365) [ClassicSimilarity], result of:
       5.512169 = fieldWeight in 4365, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.81947 = idf(docFreq=16, maxDocs=42306)
         0.625 = fieldNorm(doc=4365)
   

Similar documents (content)

1. Vilares, J.; Alonso, M.A.; Vilares, M.: Extraction of complex index terms in non-English IR : a shallow parsing based approach (2008) 0.16

   0.16286983 = sum of:
     0.16286983 = product of:
       0.6786243 = sum of:
         0.011672695 = weight(abstract_txt:that in 4108) [ClassicSimilarity], result of:
           0.011672695 = score(doc=4108,freq=3.0), product of:
             0.044837512 = queryWeight, product of:
               1.3049475 = boost
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.014287607 = queryNorm
             0.26033324 = fieldWeight in 4108, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.0625 = fieldNorm(doc=4108)
         0.086287215 = weight(abstract_txt:phrase in 4108) [ClassicSimilarity], result of:
           0.086287215 = score(doc=4108,freq=1.0), product of:
             0.19477004 = queryWeight, product of:
               1.923173 = boost
               7.088336 = idf(docFreq=95, maxDocs=42306)
               0.014287607 = queryNorm
             0.443021 = fieldWeight in 4108, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.088336 = idf(docFreq=95, maxDocs=42306)
               0.0625 = fieldNorm(doc=4108)
         0.10324341 = weight(abstract_txt:linguistic in 4108) [ClassicSimilarity], result of:
           0.10324341 = score(doc=4108,freq=2.0), product of:
             0.19944347 = queryWeight, product of:
               2.3834872 = boost
               5.8566265 = idf(docFreq=328, maxDocs=42306)
               0.014287607 = queryNorm
             0.5176575 = fieldWeight in 4108, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8566265 = idf(docFreq=328, maxDocs=42306)
               0.0625 = fieldNorm(doc=4108)
         0.1645066 = weight(abstract_txt:phrases in 4108) [ClassicSimilarity], result of:
           0.1645066 = score(doc=4108,freq=2.0), product of:
             0.27208242 = queryWeight, product of:
               2.783897 = boost
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.014287607 = queryNorm
             0.60462046 = fieldWeight in 4108, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.0625 = fieldNorm(doc=4108)
         0.24413341 = weight(abstract_txt:noun in 4108) [ClassicSimilarity], result of:
           0.24413341 = score(doc=4108,freq=2.0), product of:
             0.35399443 = queryWeight, product of:
               3.1754196 = boost
               7.8025365 = idf(docFreq=46, maxDocs=42306)
               0.014287607 = queryNorm
             0.6896533 = fieldWeight in 4108, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.8025365 = idf(docFreq=46, maxDocs=42306)
               0.0625 = fieldNorm(doc=4108)
         0.06878095 = weight(abstract_txt:terms in 4108) [ClassicSimilarity], result of:
           0.06878095 = score(doc=4108,freq=2.0), product of:
             0.1916757 = queryWeight, product of:
               3.304467 = boost
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.014287607 = queryNorm
             0.35884026 = fieldWeight in 4108, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.0625 = fieldNorm(doc=4108)
       0.24 = coord(6/25)
   

2. Bounhas, I.; Elayeb, B.; Evrard, F.; Slimani, Y.: Organizing contextual knowledge for Arabic text disambiguation and terminology extraction (2011) 0.16

   0.15965089 = sum of:
     0.15965089 = product of:
       0.66521204 = sum of:
         0.10597563 = weight(abstract_txt:nouns in 1847) [ClassicSimilarity], result of:
           0.10597563 = score(doc=1847,freq=3.0), product of:
             0.12292628 = queryWeight, product of:
               1.0803504 = boost
               7.9638047 = idf(docFreq=39, maxDocs=42306)
               0.014287607 = queryNorm
             0.86210716 = fieldWeight in 1847, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.9638047 = idf(docFreq=39, maxDocs=42306)
               0.0625 = fieldNorm(doc=1847)
         0.068602785 = weight(abstract_txt:terminology in 1847) [ClassicSimilarity], result of:
           0.068602785 = score(doc=1847,freq=1.0), product of:
             0.19134454 = queryWeight, product of:
               2.3345916 = boost
               5.736482 = idf(docFreq=370, maxDocs=42306)
               0.014287607 = queryNorm
             0.35853013 = fieldWeight in 1847, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.736482 = idf(docFreq=370, maxDocs=42306)
               0.0625 = fieldNorm(doc=1847)
         0.07300412 = weight(abstract_txt:linguistic in 1847) [ClassicSimilarity], result of:
           0.07300412 = score(doc=1847,freq=1.0), product of:
             0.19944347 = queryWeight, product of:
               2.3834872 = boost
               5.8566265 = idf(docFreq=328, maxDocs=42306)
               0.014287607 = queryNorm
             0.36603916 = fieldWeight in 1847, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8566265 = idf(docFreq=328, maxDocs=42306)
               0.0625 = fieldNorm(doc=1847)
         0.11632373 = weight(abstract_txt:phrases in 1847) [ClassicSimilarity], result of:
           0.11632373 = score(doc=1847,freq=1.0), product of:
             0.27208242 = queryWeight, product of:
               2.783897 = boost
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.014287607 = queryNorm
             0.42753124 = fieldWeight in 1847, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.0625 = fieldNorm(doc=1847)
         0.1726284 = weight(abstract_txt:noun in 1847) [ClassicSimilarity], result of:
           0.1726284 = score(doc=1847,freq=1.0), product of:
             0.35399443 = queryWeight, product of:
               3.1754196 = boost
               7.8025365 = idf(docFreq=46, maxDocs=42306)
               0.014287607 = queryNorm
             0.48765853 = fieldWeight in 1847, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.8025365 = idf(docFreq=46, maxDocs=42306)
               0.0625 = fieldNorm(doc=1847)
         0.12867737 = weight(abstract_txt:terms in 1847) [ClassicSimilarity], result of:
           0.12867737 = score(doc=1847,freq=7.0), product of:
             0.1916757 = queryWeight, product of:
               3.304467 = boost
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.014287607 = queryNorm
             0.6713286 = fieldWeight in 1847, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.0625 = fieldNorm(doc=1847)
       0.24 = coord(6/25)
   

3. Godby, J.: WordSmith research project bridges gap between tokens and indexes (1998) 0.16

   0.15816773 = sum of:
     0.15816773 = product of:
       0.6590322 = sum of:
         0.0142960735 = weight(abstract_txt:that in 5730) [ClassicSimilarity], result of:
           0.0142960735 = score(doc=5730,freq=2.0), product of:
             0.044837512 = queryWeight, product of:
               1.3049475 = boost
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.014287607 = queryNorm
             0.31884181 = fieldWeight in 5730, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.09375 = fieldNorm(doc=5730)
         0.06837351 = weight(abstract_txt:text in 5730) [ClassicSimilarity], result of:
           0.06837351 = score(doc=5730,freq=2.0), product of:
             0.12727864 = queryWeight, product of:
               2.1986194 = boost
               4.0517817 = idf(docFreq=1999, maxDocs=42306)
               0.014287607 = queryNorm
             0.53719544 = fieldWeight in 5730, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.0517817 = idf(docFreq=1999, maxDocs=42306)
               0.09375 = fieldNorm(doc=5730)
         0.1455285 = weight(abstract_txt:terminology in 5730) [ClassicSimilarity], result of:
           0.1455285 = score(doc=5730,freq=2.0), product of:
             0.19134454 = queryWeight, product of:
               2.3345916 = boost
               5.736482 = idf(docFreq=370, maxDocs=42306)
               0.014287607 = queryNorm
             0.7605573 = fieldWeight in 5730, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.736482 = idf(docFreq=370, maxDocs=42306)
               0.09375 = fieldNorm(doc=5730)
         0.1744856 = weight(abstract_txt:phrases in 5730) [ClassicSimilarity], result of:
           0.1744856 = score(doc=5730,freq=1.0), product of:
             0.27208242 = queryWeight, product of:
               2.783897 = boost
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.014287607 = queryNorm
             0.64129686 = fieldWeight in 5730, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.09375 = fieldNorm(doc=5730)
         0.07295322 = weight(abstract_txt:terms in 5730) [ClassicSimilarity], result of:
           0.07295322 = score(doc=5730,freq=1.0), product of:
             0.1916757 = queryWeight, product of:
               3.304467 = boost
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.014287607 = queryNorm
             0.38060755 = fieldWeight in 5730, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.09375 = fieldNorm(doc=5730)
         0.18339534 = weight(abstract_txt:technical in 5730) [ClassicSimilarity], result of:
           0.18339534 = score(doc=5730,freq=1.0), product of:
             0.39004004 = queryWeight, product of:
               5.443044 = boost
               5.015426 = idf(docFreq=762, maxDocs=42306)
               0.014287607 = queryNorm
             0.4701962 = fieldWeight in 5730, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.015426 = idf(docFreq=762, maxDocs=42306)
               0.09375 = fieldNorm(doc=5730)
       0.24 = coord(6/25)
   

4. Pirkola, A.; Jarvelin, K.: ¬The effect of anaphor and ellipsis resolution on proximity searching in a text database (1995) 0.15

   0.14778979 = sum of:
     0.14778979 = product of:
       0.5278207 = sum of:
         0.012130856 = weight(abstract_txt:some in 4157) [ClassicSimilarity], result of:
           0.012130856 = score(doc=4157,freq=1.0), product of:
             0.05266056 = queryWeight, product of:
               3.6857507 = idf(docFreq=2883, maxDocs=42306)
               0.014287607 = queryNorm
             0.23035942 = fieldWeight in 4157, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6857507 = idf(docFreq=2883, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
         0.061185054 = weight(abstract_txt:nouns in 4157) [ClassicSimilarity], result of:
           0.061185054 = score(doc=4157,freq=1.0), product of:
             0.12292628 = queryWeight, product of:
               1.0803504 = boost
               7.9638047 = idf(docFreq=39, maxDocs=42306)
               0.014287607 = queryNorm
             0.4977378 = fieldWeight in 4157, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.9638047 = idf(docFreq=39, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
         0.0067392336 = weight(abstract_txt:that in 4157) [ClassicSimilarity], result of:
           0.0067392336 = score(doc=4157,freq=1.0), product of:
             0.044837512 = queryWeight, product of:
               1.3049475 = boost
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.014287607 = queryNorm
             0.15030347 = fieldWeight in 4157, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
         0.086287215 = weight(abstract_txt:phrase in 4157) [ClassicSimilarity], result of:
           0.086287215 = score(doc=4157,freq=1.0), product of:
             0.19477004 = queryWeight, product of:
               1.923173 = boost
               7.088336 = idf(docFreq=95, maxDocs=42306)
               0.014287607 = queryNorm
             0.443021 = fieldWeight in 4157, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.088336 = idf(docFreq=95, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
         0.05582673 = weight(abstract_txt:text in 4157) [ClassicSimilarity], result of:
           0.05582673 = score(doc=4157,freq=3.0), product of:
             0.12727864 = queryWeight, product of:
               2.1986194 = boost
               4.0517817 = idf(docFreq=1999, maxDocs=42306)
               0.014287607 = queryNorm
             0.4386182 = fieldWeight in 4157, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.0517817 = idf(docFreq=1999, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
         0.07300412 = weight(abstract_txt:linguistic in 4157) [ClassicSimilarity], result of:
           0.07300412 = score(doc=4157,freq=1.0), product of:
             0.19944347 = queryWeight, product of:
               2.3834872 = boost
               5.8566265 = idf(docFreq=328, maxDocs=42306)
               0.014287607 = queryNorm
             0.36603916 = fieldWeight in 4157, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8566265 = idf(docFreq=328, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
         0.23264746 = weight(abstract_txt:phrases in 4157) [ClassicSimilarity], result of:
           0.23264746 = score(doc=4157,freq=4.0), product of:
             0.27208242 = queryWeight, product of:
               2.783897 = boost
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.014287607 = queryNorm
             0.8550625 = fieldWeight in 4157, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.0625 = fieldNorm(doc=4157)
       0.28 = coord(7/25)
   

5. Souza, R.R.; Raghavan, K.S.: ¬A methodology for noun phrase-based automatic indexing (2006) 0.15

   0.14771938 = sum of:
     0.14771938 = product of:
       0.7385969 = sum of:
         0.011913395 = weight(abstract_txt:that in 1299) [ClassicSimilarity], result of:
           0.011913395 = score(doc=1299,freq=2.0), product of:
             0.044837512 = queryWeight, product of:
               1.3049475 = boost
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.014287607 = queryNorm
             0.2657015 = fieldWeight in 1299, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4048555 = idf(docFreq=10381, maxDocs=42306)
               0.078125 = fieldNorm(doc=1299)
         0.040289473 = weight(abstract_txt:text in 1299) [ClassicSimilarity], result of:
           0.040289473 = score(doc=1299,freq=1.0), product of:
             0.12727864 = queryWeight, product of:
               2.1986194 = boost
               4.0517817 = idf(docFreq=1999, maxDocs=42306)
               0.014287607 = queryNorm
             0.31654543 = fieldWeight in 1299, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.0517817 = idf(docFreq=1999, maxDocs=42306)
               0.078125 = fieldNorm(doc=1299)
         0.25184825 = weight(abstract_txt:phrases in 1299) [ClassicSimilarity], result of:
           0.25184825 = score(doc=1299,freq=3.0), product of:
             0.27208242 = queryWeight, product of:
               2.783897 = boost
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.014287607 = queryNorm
             0.92563224 = fieldWeight in 1299, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.8405 = idf(docFreq=122, maxDocs=42306)
               0.078125 = fieldNorm(doc=1299)
         0.37375143 = weight(abstract_txt:noun in 1299) [ClassicSimilarity], result of:
           0.37375143 = score(doc=1299,freq=3.0), product of:
             0.35399443 = queryWeight, product of:
               3.1754196 = boost
               7.8025365 = idf(docFreq=46, maxDocs=42306)
               0.014287607 = queryNorm
             1.0558116 = fieldWeight in 1299, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               7.8025365 = idf(docFreq=46, maxDocs=42306)
               0.078125 = fieldNorm(doc=1299)
         0.06079435 = weight(abstract_txt:terms in 1299) [ClassicSimilarity], result of:
           0.06079435 = score(doc=1299,freq=1.0), product of:
             0.1916757 = queryWeight, product of:
               3.304467 = boost
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.014287607 = queryNorm
             0.31717297 = fieldWeight in 1299, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.059814 = idf(docFreq=1983, maxDocs=42306)
               0.078125 = fieldNorm(doc=1299)
       0.2 = coord(5/25)