Search (2 results, page 1 of 1)

0.04660134 = product of:
  0.09320268 = sum of:
    0.09320268 = sum of:
      0.060221836 = weight(_text_:p in 2558) [ClassicSimilarity], result of:
        0.060221836 = score(doc=2558,freq=6.0), product of:
          0.14587344 = queryWeight, product of:
            3.5955126 = idf(docFreq=3298, maxDocs=44218)
            0.04057097 = queryNorm
          0.4128362 = fieldWeight in 2558, product of:
            2.4494898 = tf(freq=6.0), with freq of:
              6.0 = termFreq=6.0
            3.5955126 = idf(docFreq=3298, maxDocs=44218)
            0.046875 = fieldNorm(doc=2558)
      0.03298084 = weight(_text_:22 in 2558) [ClassicSimilarity], result of:
        0.03298084 = score(doc=2558,freq=2.0), product of:
          0.14207263 = queryWeight, product of:
            3.5018296 = idf(docFreq=3622, maxDocs=44218)
            0.04057097 = queryNorm
          0.23214069 = fieldWeight in 2558, product of:
            1.4142135 = tf(freq=2.0), with freq of:
              2.0 = termFreq=2.0
            3.5018296 = idf(docFreq=3622, maxDocs=44218)
            0.046875 = fieldNorm(doc=2558)
  0.5 = coord(1/2)

Abstract

The paper shows that the present evaluation methods in information retrieval (basically recall R and precision P and in some cases fallout F ) lack universal comparability in the sense that their values depend on the generality of the IR problem. A solution is given by using all "parts" of the database, including the non-relevant documents and also the not-retrieved documents. It turns out that the solution is given by introducing the measure M being the fraction of the not-retrieved documents that are relevant (hence the "miss" measure). We prove that - independent of the IR problem or of the IR action - the quadruple (P,R,F,M) belongs to a universal IR surface, being the same for all IR-activities. This universality is then exploited by defining a new measure for evaluation in IR allowing for unbiased comparisons of all IR results. We also show that only using one, two or even three measures from the set {P,R,F,M} necessary leads to evaluation measures that are non-universal and hence not capable of comparing different IR situations.

Date

14. 8.2004 19:17:22

0.019436512 = product of:
  0.038873024 = sum of:
    0.038873024 = product of:
      0.07774605 = sum of:
        0.07774605 = weight(_text_:p in 2011) [ClassicSimilarity], result of:
          0.07774605 = score(doc=2011,freq=10.0), product of:
            0.14587344 = queryWeight, product of:
              3.5955126 = idf(docFreq=3298, maxDocs=44218)
              0.04057097 = queryNorm
            0.5329692 = fieldWeight in 2011, product of:
              3.1622777 = tf(freq=10.0), with freq of:
                10.0 = termFreq=10.0
              3.5955126 = idf(docFreq=3298, maxDocs=44218)
              0.046875 = fieldNorm(doc=2011)
      0.5 = coord(1/2)
  0.5 = coord(1/2)

Abstract

In an earlier paper [Egghe, L. (2004). A universal method of information retrieval evaluation: the "missing" link M and the universal IR surface. Information Processing and Management, 40, 21-30] we showed that, given an IR system, and if P denotes precision, R recall, F fallout and M miss (re-introduced in the paper mentioned above), we have the following relationship between P, R, F and M: P/(1-P)*(1-R)/R*F/(1-F)*(1-M)/M = 1. In this paper we prove the (more difficult) converse: given any four rational numbers in the interval ]0, 1[ satisfying the above equation, then there exists an IR system such that these four numbers (in any order) are the precision, recall, fallout and miss of this IR system. As a consequence we show that any three rational numbers in ]0, 1[ represent any three measures taken from precision, recall, fallout and miss of a certain IR system. We also show that this result is also true for two numbers instead of three.