Search (3 results, page 1 of 1)

0.016424898 = product of:
  0.032849796 = sum of:
    0.032849796 = product of:
      0.06569959 = sum of:
        0.06569959 = weight(_text_:n in 3301) [ClassicSimilarity], result of:
          0.06569959 = score(doc=3301,freq=4.0), product of:
            0.19504215 = queryWeight, product of:
              4.3116565 = idf(docFreq=1611, maxDocs=44218)
              0.045236014 = queryNorm
            0.33684817 = fieldWeight in 3301, product of:
              2.0 = tf(freq=4.0), with freq of:
                4.0 = termFreq=4.0
              4.3116565 = idf(docFreq=1611, maxDocs=44218)
              0.0390625 = fieldNorm(doc=3301)
      0.5 = coord(1/2)
  0.5 = coord(1/2)

Abstract

This paper describes and evaluates various stemming and indexing strategies for the Russian language. We design and evaluate two stemming approaches, a light and a more aggressive one, and compare these stemmers to the Snowball stemmer, to no stemming, and also to a language-independent approach (n-gram). To evaluate the suggested stemming strategies we apply various probabilistic information retrieval (IR) models, including the Okapi, the Divergence from Randomness (DFR), a statistical language model (LM), as well as two vector-space approaches, namely, the classical tf idf scheme and the dtu-dtn model. We find that the vector-space dtu-dtn and the DFR models tend to result in better retrieval effectiveness than the Okapi, LM, or tf idf models, while only the latter two IR approaches result in statistically significant performance differences. Ignoring stemming generally reduces the MAP by more than 50%, and these differences are always significant. When applying an n-gram approach, performance differences are usually lower than an approach involving stemming. Finally, our light stemmer tends to perform best, although performance differences between the light, aggressive, and Snowball stemmers are not statistically significant.

0.011614156 = product of:
  0.023228312 = sum of:
    0.023228312 = product of:
      0.046456624 = sum of:
        0.046456624 = weight(_text_:n in 445) [ClassicSimilarity], result of:
          0.046456624 = score(doc=445,freq=2.0), product of:
            0.19504215 = queryWeight, product of:
              4.3116565 = idf(docFreq=1611, maxDocs=44218)
              0.045236014 = queryNorm
            0.23818761 = fieldWeight in 445, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              4.3116565 = idf(docFreq=1611, maxDocs=44218)
              0.0390625 = fieldNorm(doc=445)
      0.5 = coord(1/2)
  0.5 = coord(1/2)

Abstract

To determine the author of a text's gender, various feature types have been suggested (e.g., function words, n-gram of letters, etc.) leading to a huge number of stylistic markers. To determine the target category, different machine learning models have been suggested (e.g., logistic regression, decision tree, k nearest-neighbors, support vector machine, naïve Bayes, neural networks, and random forest). In this study, our first objective is to know whether or not the same model always proposes the best effectiveness when considering similar corpora under the same conditions. Thus, based on 7 CLEF-PAN collections, this study analyzes the effectiveness of 10 different classifiers. Our second aim is to propose a 2-stage feature selection to reduce the feature size to a few hundred terms without any significant change in the performance level compared to approaches using all the attributes (increase of around 5% after applying the proposed feature selection). Based on our experiments, neural network or random forest tend, on average, to produce the highest effectiveness. Moreover, empirical evidence indicates that reducing the feature set size to around 300 without penalizing the effectiveness is possible. Finally, based on such reduced feature sizes, an analysis reveals some of the specific terms that clearly discriminate between the 2 genders.

0.0076610697 = product of:
  0.0153221395 = sum of:
    0.0153221395 = product of:
      0.030644279 = sum of:
        0.030644279 = weight(_text_:22 in 2937) [ClassicSimilarity], result of:
          0.030644279 = score(doc=2937,freq=2.0), product of:
            0.15840882 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.045236014 = queryNorm
            0.19345059 = fieldWeight in 2937, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0390625 = fieldNorm(doc=2937)
      0.5 = coord(1/2)
  0.5 = coord(1/2)

Date

7. 5.2016 21:22:27