Search (2 results, page 1 of 1)

0.018361554 = product of:
  0.073446214 = sum of:
    0.02834915 = weight(_text_:libraries in 4284) [ClassicSimilarity], result of:
      0.02834915 = score(doc=4284,freq=2.0), product of:
        0.13017908 = queryWeight, product of:
          3.2850544 = idf(docFreq=4499, maxDocs=44218)
          0.03962768 = queryNorm
        0.2177704 = fieldWeight in 4284, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          3.2850544 = idf(docFreq=4499, maxDocs=44218)
          0.046875 = fieldNorm(doc=4284)
    0.045097064 = product of:
      0.09019413 = sum of:
        0.09019413 = weight(_text_:area in 4284) [ClassicSimilarity], result of:
          0.09019413 = score(doc=4284,freq=4.0), product of:
            0.1952553 = queryWeight, product of:
              4.927245 = idf(docFreq=870, maxDocs=44218)
              0.03962768 = queryNorm
            0.46192923 = fieldWeight in 4284, product of:
              2.0 = tf(freq=4.0), with freq of:
                4.0 = termFreq=4.0
              4.927245 = idf(docFreq=870, maxDocs=44218)
              0.046875 = fieldNorm(doc=4284)
      0.5 = coord(1/2)
  0.25 = coord(2/8)

Abstract

Natural Language Processing (NLP) is an area of research and application that explores how computers can be used to understand and manipulate natural language text or speech to do useful things. NLP researchers aim to gather knowledge an how human beings understand and use language so that appropriate tools and techniques can be developed to make computer systems understand and manipulate natural languages to perform desired tasks. The foundations of NLP lie in a number of disciplines, namely, computer and information sciences, linguistics, mathematics, electrical and electronic engineering, artificial intelligence and robotics, and psychology. Applications of NLP include a number of fields of study, such as machine translation, natural language text processing and summarization, user interfaces, multilingual and cross-language information retrieval (CLIR), speech recognition, artificial intelligence, and expert systems. One important application area that is relatively new and has not been covered in previous ARIST chapters an NLP relates to the proliferation of the World Wide Web and digital libraries.

0.0033217126 = product of:
  0.0265737 = sum of:
    0.0265737 = product of:
      0.0531474 = sum of:
        0.0531474 = weight(_text_:area in 4277) [ClassicSimilarity], result of:
          0.0531474 = score(doc=4277,freq=2.0), product of:
            0.1952553 = queryWeight, product of:
              4.927245 = idf(docFreq=870, maxDocs=44218)
              0.03962768 = queryNorm
            0.27219442 = fieldWeight in 4277, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              4.927245 = idf(docFreq=870, maxDocs=44218)
              0.0390625 = fieldNorm(doc=4277)
      0.5 = coord(1/2)
  0.125 = coord(1/8)

Abstract

This chapter reviews research and applications in statistical language modeling for information retrieval (IR), which has emerged within the past several years as a new probabilistic framework for describing information retrieval processes. Generally speaking, statistical language modeling, or more simply language modeling (LM), involves estimating a probability distribution that captures statistical regularities of natural language use. Applied to information retrieval, language modeling refers to the problem of estimating the likelihood that a query and a document could have been generated by the same language model, given the language model of the document either with or without a language model of the query. The roots of statistical language modeling date to the beginning of the twentieth century when Markov tried to model letter sequences in works of Russian literature (Manning & Schütze, 1999). Zipf (1929, 1932, 1949, 1965) studied the statistical properties of text and discovered that the frequency of works decays as a Power function of each works rank. However, it was Shannon's (1951) work that inspired later research in this area. In 1951, eager to explore the applications of his newly founded information theory to human language, Shannon used a prediction game involving n-grams to investigate the information content of English text. He evaluated n-gram models' performance by comparing their crossentropy an texts with the true entropy estimated using predictions made by human subjects. For many years, statistical language models have been used primarily for automatic speech recognition. Since 1980, when the first significant language model was proposed (Rosenfeld, 2000), statistical language modeling has become a fundamental component of speech recognition, machine translation, and spelling correction.