Search (80 results, page 2 of 4)

  • × theme_ss:"Computerlinguistik"
  1. Weiß, E.-M.: ChatGPT soll es richten : Microsoft baut KI in Suchmaschine Bing ein (2023) 0.03 
    0.027065417 = product of:
  0.054130834 = sum of:
    0.054130834 = product of:
      0.10826167 = sum of:
        0.10826167 = weight(_text_:news in 866) [ClassicSimilarity], result of:
          0.10826167 = score(doc=866,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.40539116 = fieldWeight in 866, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0546875 = fieldNorm(doc=866)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Source
    https://www.heise.de/news/ChatGPT-soll-es-richten-Microsoft-baut-KI-in-Suchmaschine-Bing-ein-7447837.html
  2. McMahon, J.G.; Smith, F.J.: Improved statistical language model performance with automatic generated word hierarchies (1996) 0.02 
    0.02415974 = product of:
  0.04831948 = sum of:
    0.04831948 = product of:
      0.09663896 = sum of:
        0.09663896 = weight(_text_:22 in 3164) [ClassicSimilarity], result of:
          0.09663896 = score(doc=3164,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.5416616 = fieldWeight in 3164, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.109375 = fieldNorm(doc=3164)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Source
    Computational linguistics. 22(1996) no.2, S.217-248
  3. Ruge, G.: ¬A spreading activation network for automatic generation of thesaurus relationships (1991) 0.02 
    0.02415974 = product of:
  0.04831948 = sum of:
    0.04831948 = product of:
      0.09663896 = sum of:
        0.09663896 = weight(_text_:22 in 4506) [ClassicSimilarity], result of:
          0.09663896 = score(doc=4506,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.5416616 = fieldWeight in 4506, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.109375 = fieldNorm(doc=4506)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    8.10.2000 11:52:22
  4. Somers, H.: Example-based machine translation : Review article (1999) 0.02 
    0.02415974 = product of:
  0.04831948 = sum of:
    0.04831948 = product of:
      0.09663896 = sum of:
        0.09663896 = weight(_text_:22 in 6672) [ClassicSimilarity], result of:
          0.09663896 = score(doc=6672,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.5416616 = fieldWeight in 6672, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.109375 = fieldNorm(doc=6672)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    31. 7.1996 9:22:19
  5. New tools for human translators (1997) 0.02 
    0.02415974 = product of:
  0.04831948 = sum of:
    0.04831948 = product of:
      0.09663896 = sum of:
        0.09663896 = weight(_text_:22 in 1179) [ClassicSimilarity], result of:
          0.09663896 = score(doc=1179,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.5416616 = fieldWeight in 1179, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.109375 = fieldNorm(doc=1179)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    31. 7.1996 9:22:19
  6. Baayen, R.H.; Lieber, H.: Word frequency distributions and lexical semantics (1997) 0.02 
    0.02415974 = product of:
  0.04831948 = sum of:
    0.04831948 = product of:
      0.09663896 = sum of:
        0.09663896 = weight(_text_:22 in 3117) [ClassicSimilarity], result of:
          0.09663896 = score(doc=3117,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.5416616 = fieldWeight in 3117, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.109375 = fieldNorm(doc=3117)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    28. 2.1999 10:48:22
  7. ¬Der Student aus dem Computer (2023) 0.02 
    0.02415974 = product of:
  0.04831948 = sum of:
    0.04831948 = product of:
      0.09663896 = sum of:
        0.09663896 = weight(_text_:22 in 1079) [ClassicSimilarity], result of:
          0.09663896 = score(doc=1079,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.5416616 = fieldWeight in 1079, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.109375 = fieldNorm(doc=1079)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    27. 1.2023 16:22:55
  8. Byrne, C.C.; McCracken, S.A.: ¬An adaptive thesaurus employing semantic distance, relational inheritance and nominal compound interpretation for linguistic support of information retrieval (1999) 0.02 
    0.020708349 = product of:
  0.041416697 = sum of:
    0.041416697 = product of:
      0.082833394 = sum of:
        0.082833394 = weight(_text_:22 in 4483) [ClassicSimilarity], result of:
          0.082833394 = score(doc=4483,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.46428138 = fieldWeight in 4483, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.09375 = fieldNorm(doc=4483)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    15. 3.2000 10:22:37
  9. Boleda, G.; Evert, S.: Multiword expressions : a pain in the neck of lexical semantics (2009) 0.02 
    0.020708349 = product of:
  0.041416697 = sum of:
    0.041416697 = product of:
      0.082833394 = sum of:
        0.082833394 = weight(_text_:22 in 4888) [ClassicSimilarity], result of:
          0.082833394 = score(doc=4888,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.46428138 = fieldWeight in 4888, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.09375 = fieldNorm(doc=4888)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    1. 3.2013 14:56:22
  10. Monnerjahn, P.: Vorsprung ohne Technik : Übersetzen: Computer und Qualität (2000) 0.02 
    0.020708349 = product of:
  0.041416697 = sum of:
    0.041416697 = product of:
      0.082833394 = sum of:
        0.082833394 = weight(_text_:22 in 5429) [ClassicSimilarity], result of:
          0.082833394 = score(doc=5429,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.46428138 = fieldWeight in 5429, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.09375 = fieldNorm(doc=5429)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Source
    c't. 2000, H.22, S.230-231
  11. Li, W.; Wong, K.-F.; Yuan, C.: Toward automatic Chinese temporal information extraction (2001) 0.02 
    0.01933244 = product of:
  0.03866488 = sum of:
    0.03866488 = product of:
      0.07732976 = sum of:
        0.07732976 = weight(_text_:news in 6029) [ClassicSimilarity], result of:
          0.07732976 = score(doc=6029,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.28956512 = fieldWeight in 6029, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0390625 = fieldNorm(doc=6029)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Over the past few years, temporal information processing and temporal database management have increasingly become hot topics. Nevertheless, only a few researchers have investigated these areas in the Chinese language. This lays down the objective of our research: to exploit Chinese language processing techniques for temporal information extraction and concept reasoning. In this article, we first study the mechanism for expressing time in Chinese. On the basis of the study, we then design a general frame structure for maintaining the extracted temporal concepts and propose a system for extracting time-dependent information from Hong Kong financial news. In the system, temporal knowledge is represented by different types of temporal concepts (TTC) and different temporal relations, including absolute and relative relations, which are used to correlate between action times and reference times. In analyzing a sentence, the algorithm first determines the situation related to the verb. This in turn will identify the type of temporal concept associated with the verb. After that, the relevant temporal information is extracted and the temporal relations are derived. These relations link relevant concept frames together in chronological order, which in turn provide the knowledge to fulfill users' queries, e.g., for question-answering (i.e., Q&A) applications
  12. Khoo, C.S.G.; Dai, D.; Loh, T.E.: Using statistical and contextual information to identify two- and three-character words in Chinese text (2002) 0.02 
    0.01933244 = product of:
  0.03866488 = sum of:
    0.03866488 = product of:
      0.07732976 = sum of:
        0.07732976 = weight(_text_:news in 5206) [ClassicSimilarity], result of:
          0.07732976 = score(doc=5206,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.28956512 = fieldWeight in 5206, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0390625 = fieldNorm(doc=5206)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Khoo, Dai, and Loh examine new statistical methods for the identification of two and three character words in Chinese text. Some meaningful Chinese words are simple (independent units of one or more characters in a sentence that have independent meaning) but others are compounds of two or more simple words. In their segmentation they utilize the Modern Chinese Word Segmentation for Application of Information Processing, with some modifications to focus on meaningful words to do manual segmentation. About 37% of meaningful words are longer than 2 characters indicating a need to handle three and four character words. Four hundred sentences from news articles were manually broken into overlapping bi-grams and tri-grams. Using logistic regression, the log of the odds that such bi/tri-grams were meaningful words was calculated. Variables like relative frequency, document frequency, local frequency, and contextual and positional information, were incorporated in the model only if the concordance measure improved by at least 2% with their addition. For two- and three-character words relative frequency of adjacent characters and document frequency of overlapping bi-grams were found to be significant. Using measures of recall and precision where correct automatic segmentation is normalized either by manual segmentation or by automatic segmentation, the contextual information formula for 2 character words provides significantly better results than previous formulations and using both the 2 and 3 character formulations in combination significantly improves the 2 character results.
  13. Tseng, Y.-H.: Automatic thesaurus generation for Chinese documents (2002) 0.02 
    0.01933244 = product of:
  0.03866488 = sum of:
    0.03866488 = product of:
      0.07732976 = sum of:
        0.07732976 = weight(_text_:news in 5226) [ClassicSimilarity], result of:
          0.07732976 = score(doc=5226,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.28956512 = fieldWeight in 5226, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0390625 = fieldNorm(doc=5226)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Tseng constructs a word co-occurrence based thesaurus by means of the automatic analysis of Chinese text. Words are identified by a longest dictionary match supplemented by a key word extraction algorithm that merges back nearby tokens and accepts shorter strings of characters if they occur more often than the longest string. Single character auxiliary words are a major source of error but this can be greatly reduced with the use of a 70-character 2680 word stop list. Extracted terms with their associate document weights are sorted by decreasing frequency and the top of this list is associated using a Dice coefficient modified to account for longer documents on the weights of term pairs. Co-occurrence is not in the document as a whole but in paragraph or sentence size sections in order to reduce computation time. A window of 29 characters or 11 words was found to be sufficient. A thesaurus was produced from 25,230 Chinese news articles and judges asked to review the top 50 terms associated with each of 30 single word query terms. They determined 69% to be relevant.
  14. Multi-source, multilingual information extraction and summarization (2013) 0.02 
    0.01933244 = product of:
  0.03866488 = sum of:
    0.03866488 = product of:
      0.07732976 = sum of:
        0.07732976 = weight(_text_:news in 978) [ClassicSimilarity], result of:
          0.07732976 = score(doc=978,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.28956512 = fieldWeight in 978, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0390625 = fieldNorm(doc=978)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Information extraction (IE) and text summarization (TS) are powerful technologies for finding relevant pieces of information in text and presenting them to the user in condensed form. The ongoing information explosion makes IE and TS critical for successful functioning within the information society. These technologies face particular challenges due to the inherent multi-source nature of the information explosion. The technologies must now handle not isolated texts or individual narratives, but rather large-scale repositories and streams---in general, in multiple languages---containing a multiplicity of perspectives, opinions, or commentaries on particular topics, entities or events. There is thus a need to adapt existing techniques and develop new ones to deal with these challenges. This volume contains a selection of papers that present a variety of methodologies for content identification and extraction, as well as for content fusion and regeneration. The chapters cover various aspects of the challenges, depending on the nature of the information sought---names vs. events,--- and the nature of the sources---news streams vs. image captions vs. scientific research papers, etc. This volume aims to offer a broad and representative sample of studies from this very active research field.
  15. Sankarasubramaniam, Y.; Ramanathan, K.; Ghosh, S.: Text summarization using Wikipedia (2014) 0.02 
    0.01933244 = product of:
  0.03866488 = sum of:
    0.03866488 = product of:
      0.07732976 = sum of:
        0.07732976 = weight(_text_:news in 2693) [ClassicSimilarity], result of:
          0.07732976 = score(doc=2693,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.28956512 = fieldWeight in 2693, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0390625 = fieldNorm(doc=2693)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Automatic text summarization has been an active field of research for many years. Several approaches have been proposed, ranging from simple position and word-frequency methods, to learning and graph based algorithms. The advent of human-generated knowledge bases like Wikipedia offer a further possibility in text summarization - they can be used to understand the input text in terms of salient concepts from the knowledge base. In this paper, we study a novel approach that leverages Wikipedia in conjunction with graph-based ranking. Our approach is to first construct a bipartite sentence-concept graph, and then rank the input sentences using iterative updates on this graph. We consider several models for the bipartite graph, and derive convergence properties under each model. Then, we take up personalized and query-focused summarization, where the sentence ranks additionally depend on user interests and queries, respectively. Finally, we present a Wikipedia-based multi-document summarization algorithm. An important feature of the proposed algorithms is that they enable real-time incremental summarization - users can first view an initial summary, and then request additional content if interested. We evaluate the performance of our proposed summarizer using the ROUGE metric, and the results show that leveraging Wikipedia can significantly improve summary quality. We also present results from a user study, which suggests that using incremental summarization can help in better understanding news articles.
  16. Tao, J.; Zhou, L.; Hickey, K.: Making sense of the black-boxes : toward interpretable text classification using deep learning models (2023) 0.02 
    0.01933244 = product of:
  0.03866488 = sum of:
    0.03866488 = product of:
      0.07732976 = sum of:
        0.07732976 = weight(_text_:news in 990) [ClassicSimilarity], result of:
          0.07732976 = score(doc=990,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.28956512 = fieldWeight in 990, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.0390625 = fieldNorm(doc=990)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Text classification is a common task in data science. Despite the superior performances of deep learning based models in various text classification tasks, their black-box nature poses significant challenges for wide adoption. The knowledge-to-action framework emphasizes several principles concerning the application and use of knowledge, such as ease-of-use, customization, and feedback. With the guidance of the above principles and the properties of interpretable machine learning, we identify the design requirements for and propose an interpretable deep learning (IDeL) based framework for text classification models. IDeL comprises three main components: feature penetration, instance aggregation, and feature perturbation. We evaluate our implementation of the framework with two distinct case studies: fake news detection and social question categorization. The experiment results provide evidence for the efficacy of IDeL components in enhancing the interpretability of text classification models. Moreover, the findings are generalizable across binary and multi-label, multi-class classification problems. The proposed IDeL framework introduce a unique iField perspective for building trusted models in data science by improving the transparency and access to advanced black-box models.
  17. Hutchins, J.: From first conception to first demonstration : the nascent years of machine translation, 1947-1954. A chronology (1997) 0.02 
    0.017256958 = product of:
  0.034513917 = sum of:
    0.034513917 = product of:
      0.06902783 = sum of:
        0.06902783 = weight(_text_:22 in 1463) [ClassicSimilarity], result of:
          0.06902783 = score(doc=1463,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.38690117 = fieldWeight in 1463, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.078125 = fieldNorm(doc=1463)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    31. 7.1996 9:22:19
  18. Kuhlmann, U.; Monnerjahn, P.: Sprache auf Knopfdruck : Sieben automatische Übersetzungsprogramme im Test (2000) 0.02 
    0.017256958 = product of:
  0.034513917 = sum of:
    0.034513917 = product of:
      0.06902783 = sum of:
        0.06902783 = weight(_text_:22 in 5428) [ClassicSimilarity], result of:
          0.06902783 = score(doc=5428,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.38690117 = fieldWeight in 5428, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.078125 = fieldNorm(doc=5428)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Source
    c't. 2000, H.22, S.220-229
  19. Lezius, W.; Rapp, R.; Wettler, M.: ¬A morphology-system and part-of-speech tagger for German (1996) 0.02 
    0.017256958 = product of:
  0.034513917 = sum of:
    0.034513917 = product of:
      0.06902783 = sum of:
        0.06902783 = weight(_text_:22 in 1693) [ClassicSimilarity], result of:
          0.06902783 = score(doc=1693,freq=2.0), product of:
            0.17841205 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.05094824 = queryNorm
            0.38690117 = fieldWeight in 1693, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.078125 = fieldNorm(doc=1693)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Date
    22. 3.2015 9:37:18
  20. Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T.; Child, R.; Ramesh, A.; Ziegler, D.M.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radford, A.; Sutskever, I.; Amodei, D.: Language models are few-shot learners (2020) 0.02 
    0.015465952 = product of:
  0.030931905 = sum of:
    0.030931905 = product of:
      0.06186381 = sum of:
        0.06186381 = weight(_text_:news in 872) [ClassicSimilarity], result of:
          0.06186381 = score(doc=872,freq=2.0), product of:
            0.26705483 = queryWeight, product of:
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.05094824 = queryNorm
            0.2316521 = fieldWeight in 872, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              5.2416887 = idf(docFreq=635, maxDocs=44218)
              0.03125 = fieldNorm(doc=872)
      0.5 = coord(1/2)
  0.5 = coord(1/2)
    Abstract
    Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general.

Authors

Years

Languages

  • e 53
  • d 26
  • chi 1
  • f 1
  • m 1
  • More… Less…

Types

  • a 62
  • el 14
  • m 7
  • s 6
  • p 2
  • x 2
  • d 1
  • More… Less…

Themes

Subjects

Classifications