Document (#39683)

Author
Abdi, A.
Idris, N.
Alguliev, R.M.
Aliguliyev, R.M.
Title
Automatic summarization assessment through a combination of semantic and syntactic information for intelligent educational systems
Source
Information processing and management. 51(2015) no.4, S.340-358
Year
2015
Abstract
Summary writing is a process for creating a short version of a source text. It can be used as a measure of understanding. As grading students' summaries is a very time-consuming task, computer-assisted assessment can help teachers perform the grading more effectively. Several techniques, such as BLEU, ROUGE, N-gram co-occurrence, Latent Semantic Analysis (LSA), LSA_Ngram and LSA_ERB, have been proposed to support the automatic assessment of students' summaries. Since these techniques are more suitable for long texts, their performance is not satisfactory for the evaluation of short summaries. This paper proposes a specialized method that works well in assessing short summaries. Our proposed method integrates the semantic relations between words, and their syntactic composition. As a result, the proposed method is able to obtain high accuracy and improve the performance compared with the current techniques. Experiments have displayed that it is to be preferred over the existing techniques. A summary evaluation system based on the proposed method has also been developed.
Content
Vgl.: doi: 10.1016/j.ipm.2015.02.001.
Theme
Automatisches Abstracting

Similar documents (content)

  1. Sjöbergh, J.: Older versions of the ROUGEeval summarization evaluation system were easier to fool (2007) 0.30
    0.29912144 = sum of:
      0.29912144 = product of:
        1.2463393 = sum of:
          0.13442682 = weight(abstract_txt:summarization in 2941) [ClassicSimilarity], result of:
            0.13442682 = score(doc=2941,freq=2.0), product of:
              0.12168686 = queryWeight, product of:
                1.011939 = boost
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.016837599 = queryNorm
              1.1046946 = fieldWeight in 2941, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.109375 = fieldNorm(doc=2941)
          0.047347434 = weight(abstract_txt:evaluation in 2941) [ClassicSimilarity], result of:
            0.047347434 = score(doc=2941,freq=1.0), product of:
              0.09633942 = queryWeight, product of:
                1.273355 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.016837599 = queryNorm
              0.4914648 = fieldWeight in 2941, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.109375 = fieldNorm(doc=2941)
          0.2843483 = weight(abstract_txt:rouge in 2941) [ClassicSimilarity], result of:
            0.2843483 = score(doc=2941,freq=2.0), product of:
              0.20051806 = queryWeight, product of:
                1.2990003 = boost
                9.167778 = idf(docFreq=11, maxDocs=42306)
                0.016837599 = queryNorm
              1.4180683 = fieldWeight in 2941, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                9.167778 = idf(docFreq=11, maxDocs=42306)
                0.109375 = fieldNorm(doc=2941)
          0.10353196 = weight(abstract_txt:automatic in 2941) [ClassicSimilarity], result of:
            0.10353196 = score(doc=2941,freq=2.0), product of:
              0.12881882 = queryWeight, product of:
                1.4724386 = boost
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.016837599 = queryNorm
              0.80370206 = fieldWeight in 2941, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.109375 = fieldNorm(doc=2941)
          0.16857783 = weight(abstract_txt:method in 2941) [ClassicSimilarity], result of:
            0.16857783 = score(doc=2941,freq=3.0), product of:
              0.19623491 = queryWeight, product of:
                2.5701036 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.016837599 = queryNorm
              0.85906136 = fieldWeight in 2941, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.109375 = fieldNorm(doc=2941)
          0.50810695 = weight(abstract_txt:summaries in 2941) [ClassicSimilarity], result of:
            0.50810695 = score(doc=2941,freq=2.0), product of:
              0.46871603 = queryWeight, product of:
                3.9720747 = boost
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.016837599 = queryNorm
              1.08404 = fieldWeight in 2941, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.109375 = fieldNorm(doc=2941)
        0.24 = coord(6/25)
    
  2. Ou, S.; Khoo, S.G.; Goh, D.H.: Automatic multidocument summarization of research abstracts : design and user evaluation (2007) 0.29
    0.28500038 = sum of:
      0.28500038 = product of:
        0.89062625 = sum of:
          0.05241672 = weight(abstract_txt:integrates in 2523) [ClassicSimilarity], result of:
            0.05241672 = score(doc=2523,freq=1.0), product of:
              0.11883243 = queryWeight, product of:
                7.0575643 = idf(docFreq=98, maxDocs=42306)
                0.016837599 = queryNorm
              0.44109777 = fieldWeight in 2523, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.0575643 = idf(docFreq=98, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.07681533 = weight(abstract_txt:summarization in 2523) [ClassicSimilarity], result of:
            0.07681533 = score(doc=2523,freq=2.0), product of:
              0.12168686 = queryWeight, product of:
                1.011939 = boost
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.016837599 = queryNorm
              0.6312541 = fieldWeight in 2523, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.038262505 = weight(abstract_txt:evaluation in 2523) [ClassicSimilarity], result of:
            0.038262505 = score(doc=2523,freq=2.0), product of:
              0.09633942 = queryWeight, product of:
                1.273355 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.016837599 = queryNorm
              0.39716354 = fieldWeight in 2523, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.041833226 = weight(abstract_txt:automatic in 2523) [ClassicSimilarity], result of:
            0.041833226 = score(doc=2523,freq=1.0), product of:
              0.12881882 = queryWeight, product of:
                1.4724386 = boost
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.016837599 = queryNorm
              0.32474467 = fieldWeight in 2523, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.11397668 = weight(abstract_txt:summary in 2523) [ClassicSimilarity], result of:
            0.11397668 = score(doc=2523,freq=2.0), product of:
              0.19944926 = queryWeight, product of:
                1.8321613 = boost
                6.465298 = idf(docFreq=178, maxDocs=42306)
                0.016837599 = queryNorm
              0.571457 = fieldWeight in 2523, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.465298 = idf(docFreq=178, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.09633019 = weight(abstract_txt:method in 2523) [ClassicSimilarity], result of:
            0.09633019 = score(doc=2523,freq=3.0), product of:
              0.19623491 = queryWeight, product of:
                2.5701036 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.016837599 = queryNorm
              0.4908922 = fieldWeight in 2523, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.060379207 = weight(abstract_txt:proposed in 2523) [ClassicSimilarity], result of:
            0.060379207 = score(doc=2523,freq=1.0), product of:
              0.20728444 = queryWeight, product of:
                2.641471 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.016837599 = queryNorm
              0.29128674 = fieldWeight in 2523, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
          0.4106124 = weight(abstract_txt:summaries in 2523) [ClassicSimilarity], result of:
            0.4106124 = score(doc=2523,freq=4.0), product of:
              0.46871603 = queryWeight, product of:
                3.9720747 = boost
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.016837599 = queryNorm
              0.87603664 = fieldWeight in 2523, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.0625 = fieldNorm(doc=2523)
        0.32 = coord(8/25)
    
  3. Hirao, T.; Okumura, M.; Yasuda, N.; Isozaki, H.: Supervised automatic evaluation for summarization with voted regression model (2007) 0.26
    0.25627103 = sum of:
      0.25627103 = product of:
        0.91525364 = sum of:
          0.096019164 = weight(abstract_txt:summarization in 2943) [ClassicSimilarity], result of:
            0.096019164 = score(doc=2943,freq=2.0), product of:
              0.12168686 = queryWeight, product of:
                1.011939 = boost
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.016837599 = queryNorm
              0.7890676 = fieldWeight in 2943, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
          0.018042402 = weight(abstract_txt:been in 2943) [ClassicSimilarity], result of:
            0.018042402 = score(doc=2943,freq=1.0), product of:
              0.06337063 = queryWeight, product of:
                1.0327412 = boost
                3.6443186 = idf(docFreq=3005, maxDocs=42306)
                0.016837599 = queryNorm
              0.28471237 = fieldWeight in 2943, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.6443186 = idf(docFreq=3005, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
          0.09565626 = weight(abstract_txt:evaluation in 2943) [ClassicSimilarity], result of:
            0.09565626 = score(doc=2943,freq=8.0), product of:
              0.09633942 = queryWeight, product of:
                1.273355 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.016837599 = queryNorm
              0.99290884 = fieldWeight in 2943, product of:
                2.828427 = tf(freq=8.0), with freq of:
                  8.0 = termFreq=8.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
          0.12808758 = weight(abstract_txt:automatic in 2943) [ClassicSimilarity], result of:
            0.12808758 = score(doc=2943,freq=6.0), product of:
              0.12881882 = queryWeight, product of:
                1.4724386 = boost
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.016837599 = queryNorm
              0.9943235 = fieldWeight in 2943, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
          0.13904065 = weight(abstract_txt:method in 2943) [ClassicSimilarity], result of:
            0.13904065 = score(doc=2943,freq=4.0), product of:
              0.19623491 = queryWeight, product of:
                2.5701036 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.016837599 = queryNorm
              0.70854187 = fieldWeight in 2943, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
          0.07547401 = weight(abstract_txt:proposed in 2943) [ClassicSimilarity], result of:
            0.07547401 = score(doc=2943,freq=1.0), product of:
              0.20728444 = queryWeight, product of:
                2.641471 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.016837599 = queryNorm
              0.3641084 = fieldWeight in 2943, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
          0.36293355 = weight(abstract_txt:summaries in 2943) [ClassicSimilarity], result of:
            0.36293355 = score(doc=2943,freq=2.0), product of:
              0.46871603 = queryWeight, product of:
                3.9720747 = boost
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.016837599 = queryNorm
              0.77431434 = fieldWeight in 2943, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.078125 = fieldNorm(doc=2943)
        0.28 = coord(7/25)
    
  4. Ou, S.; Khoo, C.S.G.; Goh, D.H.: Multi-document summarization of news articles using an event-based framework (2006) 0.25
    0.24679357 = sum of:
      0.24679357 = product of:
        0.8814056 = sum of:
          0.07681533 = weight(abstract_txt:summarization in 1783) [ClassicSimilarity], result of:
            0.07681533 = score(doc=1783,freq=2.0), product of:
              0.12168686 = queryWeight, product of:
                1.011939 = boost
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.016837599 = queryNorm
              0.6312541 = fieldWeight in 1783, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
          0.038262505 = weight(abstract_txt:evaluation in 1783) [ClassicSimilarity], result of:
            0.038262505 = score(doc=1783,freq=2.0), product of:
              0.09633942 = queryWeight, product of:
                1.273355 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.016837599 = queryNorm
              0.39716354 = fieldWeight in 1783, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
          0.041833226 = weight(abstract_txt:automatic in 1783) [ClassicSimilarity], result of:
            0.041833226 = score(doc=1783,freq=1.0), product of:
              0.12881882 = queryWeight, product of:
                1.4724386 = boost
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.016837599 = queryNorm
              0.32474467 = fieldWeight in 1783, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.1959147 = idf(docFreq=636, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
          0.08059368 = weight(abstract_txt:summary in 1783) [ClassicSimilarity], result of:
            0.08059368 = score(doc=1783,freq=1.0), product of:
              0.19944926 = queryWeight, product of:
                1.8321613 = boost
                6.465298 = idf(docFreq=178, maxDocs=42306)
                0.016837599 = queryNorm
              0.40408114 = fieldWeight in 1783, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.465298 = idf(docFreq=178, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
          0.05561626 = weight(abstract_txt:method in 1783) [ClassicSimilarity], result of:
            0.05561626 = score(doc=1783,freq=1.0), product of:
              0.19623491 = queryWeight, product of:
                2.5701036 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.016837599 = queryNorm
              0.28341675 = fieldWeight in 1783, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
          0.08538909 = weight(abstract_txt:proposed in 1783) [ClassicSimilarity], result of:
            0.08538909 = score(doc=1783,freq=2.0), product of:
              0.20728444 = queryWeight, product of:
                2.641471 = boost
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.016837599 = queryNorm
              0.41194165 = fieldWeight in 1783, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.660588 = idf(docFreq=1087, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
          0.5028955 = weight(abstract_txt:summaries in 1783) [ClassicSimilarity], result of:
            0.5028955 = score(doc=1783,freq=6.0), product of:
              0.46871603 = queryWeight, product of:
                3.9720747 = boost
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.016837599 = queryNorm
              1.0729214 = fieldWeight in 1783, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.0625 = fieldNorm(doc=1783)
        0.28 = coord(7/25)
    
  5. Reeve, L.H.; Han, H.; Brooks, A.D.: ¬The use of domain-specific concepts in biomedical text summarization (2007) 0.21
    0.21494433 = sum of:
      0.21494433 = product of:
        0.76765835 = sum of:
          0.10863328 = weight(abstract_txt:summarization in 2956) [ClassicSimilarity], result of:
            0.10863328 = score(doc=2956,freq=4.0), product of:
              0.12168686 = queryWeight, product of:
                1.011939 = boost
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.016837599 = queryNorm
              0.8927281 = fieldWeight in 2956, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                7.1418247 = idf(docFreq=90, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
          0.046861805 = weight(abstract_txt:evaluation in 2956) [ClassicSimilarity], result of:
            0.046861805 = score(doc=2956,freq=3.0), product of:
              0.09633942 = queryWeight, product of:
                1.273355 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.016837599 = queryNorm
              0.486424 = fieldWeight in 2956, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
          0.11489406 = weight(abstract_txt:rouge in 2956) [ClassicSimilarity], result of:
            0.11489406 = score(doc=2956,freq=1.0), product of:
              0.20051806 = queryWeight, product of:
                1.2990003 = boost
                9.167778 = idf(docFreq=11, maxDocs=42306)
                0.016837599 = queryNorm
              0.5729861 = fieldWeight in 2956, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.167778 = idf(docFreq=11, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
          0.029959377 = weight(abstract_txt:performance in 2956) [ClassicSimilarity], result of:
            0.029959377 = score(doc=2956,freq=1.0), product of:
              0.10311463 = queryWeight, product of:
                1.3173696 = boost
                4.6487103 = idf(docFreq=1100, maxDocs=42306)
                0.016837599 = queryNorm
              0.2905444 = fieldWeight in 2956, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.6487103 = idf(docFreq=1100, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
          0.04073158 = weight(abstract_txt:semantic in 2956) [ClassicSimilarity], result of:
            0.04073158 = score(doc=2956,freq=1.0), product of:
              0.1448604 = queryWeight, product of:
                1.9123527 = boost
                4.4988503 = idf(docFreq=1278, maxDocs=42306)
                0.016837599 = queryNorm
              0.28117815 = fieldWeight in 2956, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.4988503 = idf(docFreq=1278, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
          0.13623147 = weight(abstract_txt:method in 2956) [ClassicSimilarity], result of:
            0.13623147 = score(doc=2956,freq=6.0), product of:
              0.19623491 = queryWeight, product of:
                2.5701036 = boost
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.016837599 = queryNorm
              0.69422644 = fieldWeight in 2956, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                4.534668 = idf(docFreq=1233, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
          0.29034683 = weight(abstract_txt:summaries in 2956) [ClassicSimilarity], result of:
            0.29034683 = score(doc=2956,freq=2.0), product of:
              0.46871603 = queryWeight, product of:
                3.9720747 = boost
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.016837599 = queryNorm
              0.61945146 = fieldWeight in 2956, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.008293 = idf(docFreq=103, maxDocs=42306)
                0.0625 = fieldNorm(doc=2956)
        0.28 = coord(7/25)