Wang, S.; Koopman, R.: Embed first, then predict (2019)
0.04
0.040034845 = product of:
0.08006969 = sum of:
0.060926907 = weight(_text_:digital in 5400) [ClassicSimilarity], result of:
0.060926907 = score(doc=5400,freq=4.0), product of:
0.19770671 = queryWeight, product of:
3.944552 = idf(docFreq=2326, maxDocs=44218)
0.050121464 = queryNorm
0.3081681 = fieldWeight in 5400, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.944552 = idf(docFreq=2326, maxDocs=44218)
0.0390625 = fieldNorm(doc=5400)
0.01914278 = weight(_text_:library in 5400) [ClassicSimilarity], result of:
0.01914278 = score(doc=5400,freq=2.0), product of:
0.1317883 = queryWeight, product of:
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.050121464 = queryNorm
0.14525402 = fieldWeight in 5400, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
2.6293786 = idf(docFreq=8668, maxDocs=44218)
0.0390625 = fieldNorm(doc=5400)
0.5 = coord(2/4)
- Abstract
- Automatic subject prediction is a desirable feature for modern digital library systems, as manual indexing can no longer cope with the rapid growth of digital collections. It is also desirable to be able to identify a small set of entities (e.g., authors, citations, bibliographic records) which are most relevant to a query. This gets more difficult when the amount of data increases dramatically. Data sparsity and model scalability are the major challenges to solving this type of extreme multilabel classification problem automatically. In this paper, we propose to address this problem in two steps: we first embed different types of entities into the same semantic space, where similarity could be computed easily; second, we propose a novel non-parametric method to identify the most relevant entities in addition to direct semantic similarities. We show how effectively this approach predicts even very specialised subjects, which are associated with few documents in the training set and are more problematic for a classifier.
