-
Hotho, A.; Bloehdorn, S.: Data Mining 2004 : Text classification by boosting weak learners based on terms and concepts (2004)
0.16
0.1618816 = sum of:
0.08274635 = product of:
0.24823904 = sum of:
0.24823904 = weight(_text_:3a in 562) [ClassicSimilarity], result of:
0.24823904 = score(doc=562,freq=2.0), product of:
0.4416923 = queryWeight, product of:
8.478011 = idf(docFreq=24, maxDocs=44218)
0.052098576 = queryNorm
0.56201804 = fieldWeight in 562, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
8.478011 = idf(docFreq=24, maxDocs=44218)
0.046875 = fieldNorm(doc=562)
0.33333334 = coord(1/3)
0.07913525 = sum of:
0.03678342 = weight(_text_:web in 562) [ClassicSimilarity], result of:
0.03678342 = score(doc=562,freq=2.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.21634221 = fieldWeight in 562, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=562)
0.042351827 = weight(_text_:22 in 562) [ClassicSimilarity], result of:
0.042351827 = score(doc=562,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.23214069 = fieldWeight in 562, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=562)
- Content
- Vgl.: http://www.google.de/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CEAQFjAA&url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.91.4940%26rep%3Drep1%26type%3Dpdf&ei=dOXrUMeIDYHDtQahsIGACg&usg=AFQjCNHFWVh6gNPvnOrOS9R3rkrXCNVD-A&sig2=5I2F5evRfMnsttSgFF9g7Q&bvm=bv.1357316858,d.Yms.
- Date
- 8. 1.2013 10:22:32
-
Huo, W.: Automatic multi-word term extraction and its application to Web-page summarization (2012)
0.06
0.057959333 = product of:
0.115918666 = sum of:
0.115918666 = sum of:
0.07356684 = weight(_text_:web in 563) [ClassicSimilarity], result of:
0.07356684 = score(doc=563,freq=8.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.43268442 = fieldWeight in 563, product of:
2.828427 = tf(freq=8.0), with freq of:
8.0 = termFreq=8.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=563)
0.042351827 = weight(_text_:22 in 563) [ClassicSimilarity], result of:
0.042351827 = score(doc=563,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.23214069 = fieldWeight in 563, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=563)
0.5 = coord(1/2)
- Abstract
- In this thesis we propose three new word association measures for multi-word term extraction. We combine these association measures with LocalMaxs algorithm in our extraction model and compare the results of different multi-word term extraction methods. Our approach is language and domain independent and requires no training data. It can be applied to such tasks as text summarization, information retrieval, and document classification. We further explore the potential of using multi-word terms as an effective representation for general web-page summarization. We extract multi-word terms from human written summaries in a large collection of web-pages, and generate the summaries by aligning document words with these multi-word terms. Our system applies machine translation technology to learn the aligning process from a training set and focuses on selecting high quality multi-word terms from human written summaries to generate suitable results for web-page summarization.
- Date
- 10. 1.2013 19:22:47
-
Doszkocs, T.E.; Zamora, A.: Dictionary services and spelling aids for Web searching (2004)
0.05
0.0515022 = product of:
0.1030044 = sum of:
0.1030044 = sum of:
0.053092297 = weight(_text_:web in 2541) [ClassicSimilarity], result of:
0.053092297 = score(doc=2541,freq=6.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.3122631 = fieldWeight in 2541, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0390625 = fieldNorm(doc=2541)
0.049912106 = weight(_text_:22 in 2541) [ClassicSimilarity], result of:
0.049912106 = score(doc=2541,freq=4.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.27358043 = fieldWeight in 2541, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.0390625 = fieldNorm(doc=2541)
0.5 = coord(1/2)
- Abstract
- The Specialized Information Services Division (SIS) of the National Library of Medicine (NLM) provides Web access to more than a dozen scientific databases on toxicology and the environment on TOXNET . Search queries on TOXNET often include misspelled or variant English words, medical and scientific jargon and chemical names. Following the example of search engines like Google and ClinicalTrials.gov, we set out to develop a spelling "suggestion" system for increased recall and precision in TOXNET searching. This paper describes development of dictionary technology that can be used in a variety of applications such as orthographic verification, writing aid, natural language processing, and information storage and retrieval. The design of the technology allows building complex applications using the components developed in the earlier phases of the work in a modular fashion without extensive rewriting of computer code. Since many of the potential applications envisioned for this work have on-line or web-based interfaces, the dictionaries and other computer components must have fast response, and must be adaptable to open-ended database vocabularies, including chemical nomenclature. The dictionary vocabulary for this work was derived from SIS and other databases and specialized resources, such as NLM's Unified Medical Language Systems (UMLS) . The resulting technology, A-Z Dictionary (AZdict), has three major constituents: 1) the vocabulary list, 2) the word attributes that define part of speech and morphological relationships between words in the list, and 3) a set of programs that implements the retrieval of words and their attributes, and determines similarity between words (ChemSpell). These three components can be used in various applications such as spelling verification, spelling aid, part-of-speech tagging, paraphrasing, and many other natural language processing functions.
- Date
- 14. 8.2004 17:22:56
- Source
- Online. 28(2004) no.3, S.22-29
-
Bian, G.-W.; Chen, H.-H.: Cross-language information access to multilingual collections on the Internet (2000)
0.05
0.04718572 = product of:
0.09437144 = sum of:
0.09437144 = sum of:
0.052019615 = weight(_text_:web in 4436) [ClassicSimilarity], result of:
0.052019615 = score(doc=4436,freq=4.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.3059541 = fieldWeight in 4436, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=4436)
0.042351827 = weight(_text_:22 in 4436) [ClassicSimilarity], result of:
0.042351827 = score(doc=4436,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.23214069 = fieldWeight in 4436, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.046875 = fieldNorm(doc=4436)
0.5 = coord(1/2)
- Abstract
- Language barrier is the major problem that people face in searching for, retrieving, and understanding multilingual collections on the Internet. This paper deals with query translation and document translation in a Chinese-English information retrieval system called MTIR. Bilingual dictionary and monolingual corpus-based approaches are adopted to select suitable tranlated query terms. A machine transliteration algorithm is introduced to resolve proper name searching. We consider several design issues for document translation, including which material is translated, what roles the HTML tags play in translation, what the tradeoff is between the speed performance and the translation performance, and what from the translated result is presented in. About 100.000 Web pages translated in the last 4 months of 1997 are used for quantitative study of online and real-time Web page translation
- Date
- 16. 2.2000 14:22:39
-
Noever, D.; Ciolino, M.: ¬The Turing deception (2022)
0.04
0.041373175 = product of:
0.08274635 = sum of:
0.08274635 = product of:
0.24823904 = sum of:
0.24823904 = weight(_text_:3a in 862) [ClassicSimilarity], result of:
0.24823904 = score(doc=862,freq=2.0), product of:
0.4416923 = queryWeight, product of:
8.478011 = idf(docFreq=24, maxDocs=44218)
0.052098576 = queryNorm
0.56201804 = fieldWeight in 862, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
8.478011 = idf(docFreq=24, maxDocs=44218)
0.046875 = fieldNorm(doc=862)
0.33333334 = coord(1/3)
0.5 = coord(1/2)
- Source
- https%3A%2F%2Farxiv.org%2Fabs%2F2212.06721&usg=AOvVaw3i_9pZm9y_dQWoHi6uv0EN
-
Yang, C.C.; Luk, J.: Automatic generation of English/Chinese thesaurus based on a parallel corpus in laws (2003)
0.03
0.033809613 = product of:
0.06761923 = sum of:
0.06761923 = sum of:
0.042913996 = weight(_text_:web in 1616) [ClassicSimilarity], result of:
0.042913996 = score(doc=1616,freq=8.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.25239927 = fieldWeight in 1616, product of:
2.828427 = tf(freq=8.0), with freq of:
8.0 = termFreq=8.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.02734375 = fieldNorm(doc=1616)
0.024705233 = weight(_text_:22 in 1616) [ClassicSimilarity], result of:
0.024705233 = score(doc=1616,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.1354154 = fieldWeight in 1616, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.02734375 = fieldNorm(doc=1616)
0.5 = coord(1/2)
- Abstract
- The information available in languages other than English in the World Wide Web is increasing significantly. According to a report from Computer Economics in 1999, 54% of Internet users are English speakers ("English Will Dominate Web for Only Three More Years," Computer Economics, July 9, 1999, http://www.computereconomics. com/new4/pr/pr990610.html). However, it is predicted that there will be only 60% increase in Internet users among English speakers verses a 150% growth among nonEnglish speakers for the next five years. By 2005, 57% of Internet users will be non-English speakers. A report by CNN.com in 2000 showed that the number of Internet users in China had been increased from 8.9 million to 16.9 million from January to June in 2000 ("Report: China Internet users double to 17 million," CNN.com, July, 2000, http://cnn.org/2000/TECH/computing/07/27/ china.internet.reut/index.html). According to Nielsen/ NetRatings, there was a dramatic leap from 22.5 millions to 56.6 millions Internet users from 2001 to 2002. China had become the second largest global at-home Internet population in 2002 (US's Internet population was 166 millions) (Robyn Greenspan, "China Pulls Ahead of Japan," Internet.com, April 22, 2002, http://cyberatias.internet.com/big-picture/geographics/article/0,,5911_1013841,00. html). All of the evidences reveal the importance of crosslingual research to satisfy the needs in the near future. Digital library research has been focusing in structural and semantic interoperability in the past. Searching and retrieving objects across variations in protocols, formats and disciplines are widely explored (Schatz, B., & Chen, H. (1999). Digital libraries: technological advances and social impacts. IEEE Computer, Special Issue an Digital Libraries, February, 32(2), 45-50.; Chen, H., Yen, J., & Yang, C.C. (1999). International activities: development of Asian digital libraries. IEEE Computer, Special Issue an Digital Libraries, 32(2), 48-49.). However, research in crossing language boundaries, especially across European languages and Oriental languages, is still in the initial stage. In this proposal, we put our focus an cross-lingual semantic interoperability by developing automatic generation of a cross-lingual thesaurus based an English/Chinese parallel corpus. When the searchers encounter retrieval problems, Professional librarians usually consult the thesaurus to identify other relevant vocabularies. In the problem of searching across language boundaries, a cross-lingual thesaurus, which is generated by co-occurrence analysis and Hopfield network, can be used to generate additional semantically relevant terms that cannot be obtained from dictionary. In particular, the automatically generated cross-lingual thesaurus is able to capture the unknown words that do not exist in a dictionary, such as names of persons, organizations, and events. Due to Hong Kong's unique history background, both English and Chinese are used as official languages in all legal documents. Therefore, English/Chinese cross-lingual information retrieval is critical for applications in courts and the government. In this paper, we develop an automatic thesaurus by the Hopfield network based an a parallel corpus collected from the Web site of the Department of Justice of the Hong Kong Special Administrative Region (HKSAR) Government. Experiments are conducted to measure the precision and recall of the automatic generated English/Chinese thesaurus. The result Shows that such thesaurus is a promising tool to retrieve relevant terms, especially in the language that is not the same as the input term. The direct translation of the input term can also be retrieved in most of the cases.
- Footnote
- Teil eines Themenheftes: "Web retrieval and mining: A machine learning perspective"
-
Warner, A.J.: Natural language processing (1987)
0.03
0.028234553 = product of:
0.056469105 = sum of:
0.056469105 = product of:
0.11293821 = sum of:
0.11293821 = weight(_text_:22 in 337) [ClassicSimilarity], result of:
0.11293821 = score(doc=337,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.61904186 = fieldWeight in 337, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.125 = fieldNorm(doc=337)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Source
- Annual review of information science and technology. 22(1987), S.79-108
-
Rettinger, A.; Schumilin, A.; Thoma, S.; Ell, B.: Learning a cross-lingual semantic representation of relations expressed in text (2015)
0.03
0.026546149 = product of:
0.053092297 = sum of:
0.053092297 = product of:
0.106184594 = sum of:
0.106184594 = weight(_text_:web in 2027) [ClassicSimilarity], result of:
0.106184594 = score(doc=2027,freq=6.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.6245262 = fieldWeight in 2027, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.078125 = fieldNorm(doc=2027)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Series
- Information Systems and Applications, incl. Internet/Web, and HCI; Bd. 9088
- Source
- The Semantic Web: latest advances and new domains. 12th European Semantic Web Conference, ESWC 2015 Portoroz, Slovenia, May 31 -- June 4, 2015. Proceedings. Eds.: F. Gandon u.a
-
McMahon, J.G.; Smith, F.J.: Improved statistical language model performance with automatic generated word hierarchies (1996)
0.02
0.024705233 = product of:
0.049410466 = sum of:
0.049410466 = product of:
0.09882093 = sum of:
0.09882093 = weight(_text_:22 in 3164) [ClassicSimilarity], result of:
0.09882093 = score(doc=3164,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.5416616 = fieldWeight in 3164, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.109375 = fieldNorm(doc=3164)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Source
- Computational linguistics. 22(1996) no.2, S.217-248
-
Ruge, G.: ¬A spreading activation network for automatic generation of thesaurus relationships (1991)
0.02
0.024705233 = product of:
0.049410466 = sum of:
0.049410466 = product of:
0.09882093 = sum of:
0.09882093 = weight(_text_:22 in 4506) [ClassicSimilarity], result of:
0.09882093 = score(doc=4506,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.5416616 = fieldWeight in 4506, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.109375 = fieldNorm(doc=4506)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 8.10.2000 11:52:22
-
Somers, H.: Example-based machine translation : Review article (1999)
0.02
0.024705233 = product of:
0.049410466 = sum of:
0.049410466 = product of:
0.09882093 = sum of:
0.09882093 = weight(_text_:22 in 6672) [ClassicSimilarity], result of:
0.09882093 = score(doc=6672,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.5416616 = fieldWeight in 6672, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.109375 = fieldNorm(doc=6672)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 31. 7.1996 9:22:19
-
New tools for human translators (1997)
0.02
0.024705233 = product of:
0.049410466 = sum of:
0.049410466 = product of:
0.09882093 = sum of:
0.09882093 = weight(_text_:22 in 1179) [ClassicSimilarity], result of:
0.09882093 = score(doc=1179,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.5416616 = fieldWeight in 1179, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.109375 = fieldNorm(doc=1179)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 31. 7.1996 9:22:19
-
Baayen, R.H.; Lieber, H.: Word frequency distributions and lexical semantics (1997)
0.02
0.024705233 = product of:
0.049410466 = sum of:
0.049410466 = product of:
0.09882093 = sum of:
0.09882093 = weight(_text_:22 in 3117) [ClassicSimilarity], result of:
0.09882093 = score(doc=3117,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.5416616 = fieldWeight in 3117, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.109375 = fieldNorm(doc=3117)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 28. 2.1999 10:48:22
-
Byrne, C.C.; McCracken, S.A.: ¬An adaptive thesaurus employing semantic distance, relational inheritance and nominal compound interpretation for linguistic support of information retrieval (1999)
0.02
0.021175914 = product of:
0.042351827 = sum of:
0.042351827 = product of:
0.084703654 = sum of:
0.084703654 = weight(_text_:22 in 4483) [ClassicSimilarity], result of:
0.084703654 = score(doc=4483,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.46428138 = fieldWeight in 4483, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.09375 = fieldNorm(doc=4483)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 15. 3.2000 10:22:37
-
Boleda, G.; Evert, S.: Multiword expressions : a pain in the neck of lexical semantics (2009)
0.02
0.021175914 = product of:
0.042351827 = sum of:
0.042351827 = product of:
0.084703654 = sum of:
0.084703654 = weight(_text_:22 in 4888) [ClassicSimilarity], result of:
0.084703654 = score(doc=4888,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.46428138 = fieldWeight in 4888, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.09375 = fieldNorm(doc=4888)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 1. 3.2013 14:56:22
-
Hutchins, J.: From first conception to first demonstration : the nascent years of machine translation, 1947-1954. A chronology (1997)
0.02
0.017646596 = product of:
0.03529319 = sum of:
0.03529319 = product of:
0.07058638 = sum of:
0.07058638 = weight(_text_:22 in 1463) [ClassicSimilarity], result of:
0.07058638 = score(doc=1463,freq=2.0), product of:
0.18244034 = queryWeight, product of:
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.052098576 = queryNorm
0.38690117 = fieldWeight in 1463, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.5018296 = idf(docFreq=3622, maxDocs=44218)
0.078125 = fieldNorm(doc=1463)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Date
- 31. 7.1996 9:22:19
-
Wang, F.L.; Yang, C.C.: Mining Web data for Chinese segmentation (2007)
0.02
0.017135466 = product of:
0.03427093 = sum of:
0.03427093 = product of:
0.06854186 = sum of:
0.06854186 = weight(_text_:web in 604) [ClassicSimilarity], result of:
0.06854186 = score(doc=604,freq=10.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.40312994 = fieldWeight in 604, product of:
3.1622777 = tf(freq=10.0), with freq of:
10.0 = termFreq=10.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.0390625 = fieldNorm(doc=604)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Modern information retrieval systems use keywords within documents as indexing terms for search of relevant documents. As Chinese is an ideographic character-based language, the words in the texts are not delimited by white spaces. Indexing of Chinese documents is impossible without a proper segmentation algorithm. Many Chinese segmentation algorithms have been proposed in the past. Traditional segmentation algorithms cannot operate without a large dictionary or a large corpus of training data. Nowadays, the Web has become the largest corpus that is ideal for Chinese segmentation. Although most search engines have problems in segmenting texts into proper words, they maintain huge databases of documents and frequencies of character sequences in the documents. Their databases are important potential resources for segmentation. In this paper, we propose a segmentation algorithm by mining Web data with the help of search engines. On the other hand, the Romanized pinyin of Chinese language indicates boundaries of words in the text. Our algorithm is the first to utilize the Romanized pinyin to segmentation. It is the first unified segmentation algorithm for the Chinese language from different geographical areas, and it is also domain independent because of the nature of the Web. Experiments have been conducted on the datasets of a recent Chinese segmentation competition. The results show that our algorithm outperforms the traditional algorithms in terms of precision and recall. Moreover, our algorithm can effectively deal with the problems of segmentation ambiguity, new word (unknown word) detection, and stop words.
- Footnote
- Beitrag eines Themenschwerpunktes "Mining Web resources for enhancing information retrieval"
-
Radev, D.; Fan, W.; Qu, H.; Wu, H.; Grewal, A.: Probabilistic question answering on the Web (2005)
0.02
0.015927691 = product of:
0.031855382 = sum of:
0.031855382 = product of:
0.063710764 = sum of:
0.063710764 = weight(_text_:web in 3455) [ClassicSimilarity], result of:
0.063710764 = score(doc=3455,freq=6.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.37471575 = fieldWeight in 3455, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=3455)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Web-based search engines such as Google and NorthernLight return documents that are relevant to a user query, not answers to user questions. We have developed an architecture that augments existing search engines so that they support natural language question answering. The process entails five steps: query modulation, document retrieval, passage extraction, phrase extraction, and answer ranking. In this article, we describe some probabilistic approaches to the last three of these stages. We show how our techniques apply to a number of existing search engines, and we also present results contrasting three different methods for question answering. Our algorithm, probabilistic phrase reranking (PPR), uses proximity and question type features and achieves a total reciprocal document rank of .20 an the TREC8 corpus. Our techniques have been implemented as a Web-accessible system, called NSIR.
-
Thelwall, M.; Price, L.: Language evolution and the spread of ideas on the Web : a procedure for identifying emergent hybrid word (2006)
0.02
0.015927691 = product of:
0.031855382 = sum of:
0.031855382 = product of:
0.063710764 = sum of:
0.063710764 = weight(_text_:web in 5896) [ClassicSimilarity], result of:
0.063710764 = score(doc=5896,freq=6.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.37471575 = fieldWeight in 5896, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=5896)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Word usage is of interest to linguists for its own sake as well as to social scientists and others who seek to track the spread of ideas, for example, in public debates over political decisions. The historical evolution of language can be analyzed with the tools of corpus linguistics through evolving corpora and the Web. But word usage statistics can only be gathered for known words. In this article, techniques are described and tested for identifying new words from the Web, focusing on the case when the words are related to a topic and have a hybrid form with a common sequence of letters. The results highlight the need to employ a combination of search techniques and show the wide potential of hybrid word family investigations in linguistics and social science.
-
Airio, E.: Who benefits from CLIR in web retrieval? (2008)
0.02
0.015927691 = product of:
0.031855382 = sum of:
0.031855382 = product of:
0.063710764 = sum of:
0.063710764 = weight(_text_:web in 2342) [ClassicSimilarity], result of:
0.063710764 = score(doc=2342,freq=6.0), product of:
0.17002425 = queryWeight, product of:
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.052098576 = queryNorm
0.37471575 = fieldWeight in 2342, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
3.2635105 = idf(docFreq=4597, maxDocs=44218)
0.046875 = fieldNorm(doc=2342)
0.5 = coord(1/2)
0.5 = coord(1/2)
- Abstract
- Purpose - The aim of the current paper is to test whether query translation is beneficial in web retrieval. Design/methodology/approach - The language pairs were Finnish-Swedish, English-German and Finnish-French. A total of 12-18 participants were recruited for each language pair. Each participant performed four retrieval tasks. The author's aim was to compare the performance of the translated queries with that of the target language queries. Thus, the author asked participants to formulate a source language query and a target language query for each task. The source language queries were translated into the target language utilizing a dictionary-based system. In English-German, also machine translation was utilized. The author used Google as the search engine. Findings - The results differed depending on the language pair. The author concluded that the dictionary coverage had an effect on the results. On average, the results of query-translation were better than in the traditional laboratory tests. Originality/value - This research shows that query translation in web is beneficial especially for users with moderate and non-active language skills. This is valuable information for developers of cross-language information retrieval systems.
