Dolamic, L.; Savoy, J.: When stopword lists make the difference (2009)
0.02
0.015656501 = product of:
0.0469695 = sum of:
0.0469695 = weight(_text_:search in 3319) [ClassicSimilarity], result of:
0.0469695 = score(doc=3319,freq=2.0), product of:
0.1747324 = queryWeight, product of:
3.475677 = idf(docFreq=3718, maxDocs=44218)
0.05027291 = queryNorm
0.2688082 = fieldWeight in 3319, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
3.475677 = idf(docFreq=3718, maxDocs=44218)
0.0546875 = fieldNorm(doc=3319)
0.33333334 = coord(1/3)
- Abstract
- In this brief communication, we evaluate the use of two stopword lists for the English language (one comprising 571 words and another with 9) and compare them with a search approach accounting for all word forms. We show that through implementing the original Okapi form or certain ones derived from the Divergence from Randomness (DFR) paradigm, significantly lower performance levels may result when using short or no stopword lists. For other DFR models and a revised Okapi implementation, performance differences between approaches using short or long stopword lists or no list at all are usually not statistically significant. Similar conclusions can be drawn when using other natural languages such as French, Hindi, or Persian.
