Search (24 results, page 2 of 2)

0.0026484418 = product of:
  0.010593767 = sum of:
    0.010593767 = product of:
      0.0317813 = sum of:
        0.0317813 = weight(_text_:k in 945) [ClassicSimilarity], result of:
          0.0317813 = score(doc=945,freq=2.0), product of:
            0.13429943 = queryWeight, product of:
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.037621226 = queryNorm
            0.23664509 = fieldWeight in 945, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.046875 = fieldNorm(doc=945)
      0.33333334 = coord(1/3)
  0.25 = coord(1/4)

Abstract

The paper focuses on a particular approach to automatic sentence compression which makes use of a discriminative sequence classifier known as Conditional Random Fields (CRF). We devise several features for CRF that allow it to incorporate information on nonlinear relations among words. Along with that, we address the issue of data paucity by collecting data from RSS feeds available on the Internet, and turning them into training data for use with CRF, drawing on techniques from biology and information retrieval. We also discuss a recursive application of CRF on the syntactic structure of a sentence as a way of improving the readability of the compression it generates. Experiments found that our approach works reasonably well compared to the state-of-the-art system [Knight, K., & Marcu, D. (2002). Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artificial Intelligence 139, 91-107.].

0.0026484418 = product of:
  0.010593767 = sum of:
    0.010593767 = product of:
      0.0317813 = sum of:
        0.0317813 = weight(_text_:k in 949) [ClassicSimilarity], result of:
          0.0317813 = score(doc=949,freq=2.0), product of:
            0.13429943 = queryWeight, product of:
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.037621226 = queryNorm
            0.23664509 = fieldWeight in 949, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.046875 = fieldNorm(doc=949)
      0.33333334 = coord(1/3)
  0.25 = coord(1/4)

0.0022070347 = product of:
  0.008828139 = sum of:
    0.008828139 = product of:
      0.026484415 = sum of:
        0.026484415 = weight(_text_:k in 2693) [ClassicSimilarity], result of:
          0.026484415 = score(doc=2693,freq=2.0), product of:
            0.13429943 = queryWeight, product of:
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.037621226 = queryNorm
            0.19720423 = fieldWeight in 2693, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.0390625 = fieldNorm(doc=2693)
      0.33333334 = coord(1/3)
  0.25 = coord(1/4)

0.0017656278 = product of:
  0.007062511 = sum of:
    0.007062511 = product of:
      0.021187533 = sum of:
        0.021187533 = weight(_text_:k in 947) [ClassicSimilarity], result of:
          0.021187533 = score(doc=947,freq=2.0), product of:
            0.13429943 = queryWeight, product of:
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.037621226 = queryNorm
            0.15776339 = fieldWeight in 947, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.569778 = idf(docFreq=3384, maxDocs=44218)
              0.03125 = fieldNorm(doc=947)
      0.33333334 = coord(1/3)
  0.25 = coord(1/4)

Abstract

Information retrieval systems consist of many complicated components. Research and development of such systems is often hampered by the difficulty in evaluating how each particular component would behave across multiple systems. We present a novel integrated information retrieval system-the Query, Cluster, Summarize (QCS) system-which is portable, modular, and permits experimentation with different instantiations of each of the constituent text analysis components. Most importantly, the combination of the three types of methods in the QCS design improves retrievals by providing users more focused information organized by topic. We demonstrate the improved performance by a series of experiments using standard test sets from the Document Understanding Conferences (DUC) as measured by the best known automatic metric for summarization system evaluation, ROUGE. Although the DUC data and evaluations were originally designed to test multidocument summarization, we developed a framework to extend it to the task of evaluation for each of the three components: query, clustering, and summarization. Under this framework, we then demonstrate that the QCS system (end-to-end) achieves performance as good as or better than the best summarization engines. Given a query, QCS retrieves relevant documents, separates the retrieved documents into topic clusters, and creates a single summary for each cluster. In the current implementation, Latent Semantic Indexing is used for retrieval, generalized spherical k-means is used for the document clustering, and a method coupling sentence "trimming" and a hidden Markov model, followed by a pivoted QR decomposition, is used to create a single extract summary for each cluster. The user interface is designed to provide access to detailed information in a compact and useful format. Our system demonstrates the feasibility of assembling an effective IR system from existing software libraries, the usefulness of the modularity of the design, and the value of this particular combination of modules.