Document (#39272)

Author
Mutz, R.
Bornmann, L.
Daniel, H.-D.
Title
Testing for the fairness and predictive validity of research funding decisions : a multilevel multiple imputation for missing data approach using ex-ante and ex-post peer evaluation data from the Austrian science fund
Source
Journal of the Association for Information Science and Technology. 66(2015) no.11, S.2321-2339
Year
2015
Abstract
It is essential for research funding organizations to ensure both the validity and fairness of the grant approval procedure. The ex-ante peer evaluation (EXANTE) of N?=?8,496 grant applications submitted to the Austrian Science Fund from 1999 to 2009 was statistically analyzed. For 1,689 funded research projects an ex-post peer evaluation (EXPOST) was also available; for the rest of the grant applications a multilevel missing data imputation approach was used to consider verification bias for the first time in peer-review research. Without imputation, the predictive validity of EXANTE was low (r?=?.26) but underestimated due to verification bias, and with imputation it was r?=?.49. That is, the decision-making procedure is capable of selecting the best research proposals for funding. In the EXANTE there were several potential biases (e.g., gender). With respect to the EXPOST there was only one real bias (discipline-specific and year-specific differential prediction). The novelty of this contribution is, first, the combining of theoretical concepts of validity and fairness with a missing data imputation approach to correct for verification bias and, second, multilevel modeling to test peer review-based funding decisions for both validity and fairness in terms of potential and real biases.
Content
Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23315/abstract.

Similar documents (author)

  1. Bornmann, L.; Mutz, R.; Daniel, H.-D.: Are there better indices for evaluation purposes than the h index? : a comparison of nine different variants of the h index using data from biomedicine (2008) 5.40
    5.3989544 = sum of:
      5.3989544 = sum of:
        1.3063266 = weight(author_txt:bornmann in 3609) [ClassicSimilarity], result of:
          1.3063266 = score(doc=3609,freq=1.0), product of:
            0.46289477 = queryWeight, product of:
              7.52555 = idf(docFreq=61, maxDocs=42306)
              0.06150976 = queryNorm
            2.822081 = fieldWeight in 3609, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              7.52555 = idf(docFreq=61, maxDocs=42306)
              0.375 = fieldNorm(doc=3609)
        1.6630177 = weight(author_txt:daniel in 3609) [ClassicSimilarity], result of:
          1.6630177 = score(doc=3609,freq=1.0), product of:
            0.54372466 = queryWeight, product of:
              1.083798 = boost
              8.156177 = idf(docFreq=32, maxDocs=42306)
              0.06150976 = queryNorm
            3.058566 = fieldWeight in 3609, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.156177 = idf(docFreq=32, maxDocs=42306)
              0.375 = fieldNorm(doc=3609)
        2.4296103 = weight(author_txt:mutz in 3609) [ClassicSimilarity], result of:
          2.4296103 = score(doc=3609,freq=1.0), product of:
            0.70006573 = queryWeight, product of:
              1.2297825 = boost
              9.254789 = idf(docFreq=10, maxDocs=42306)
              0.06150976 = queryNorm
            3.470546 = fieldWeight in 3609, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.254789 = idf(docFreq=10, maxDocs=42306)
              0.375 = fieldNorm(doc=3609)
    
  2. Bornmann, L.; Mutz, R.; Daniel, H.D.: Do we need the h index and its variants in addition to standard bibliometric measures? (2009) 5.40
    5.3989544 = sum of:
      5.3989544 = sum of:
        1.3063266 = weight(author_txt:bornmann in 681) [ClassicSimilarity], result of:
          1.3063266 = score(doc=681,freq=1.0), product of:
            0.46289477 = queryWeight, product of:
              7.52555 = idf(docFreq=61, maxDocs=42306)
              0.06150976 = queryNorm
            2.822081 = fieldWeight in 681, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              7.52555 = idf(docFreq=61, maxDocs=42306)
              0.375 = fieldNorm(doc=681)
        1.6630177 = weight(author_txt:daniel in 681) [ClassicSimilarity], result of:
          1.6630177 = score(doc=681,freq=1.0), product of:
            0.54372466 = queryWeight, product of:
              1.083798 = boost
              8.156177 = idf(docFreq=32, maxDocs=42306)
              0.06150976 = queryNorm
            3.058566 = fieldWeight in 681, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.156177 = idf(docFreq=32, maxDocs=42306)
              0.375 = fieldNorm(doc=681)
        2.4296103 = weight(author_txt:mutz in 681) [ClassicSimilarity], result of:
          2.4296103 = score(doc=681,freq=1.0), product of:
            0.70006573 = queryWeight, product of:
              1.2297825 = boost
              9.254789 = idf(docFreq=10, maxDocs=42306)
              0.06150976 = queryNorm
            3.470546 = fieldWeight in 681, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.254789 = idf(docFreq=10, maxDocs=42306)
              0.375 = fieldNorm(doc=681)
    
  3. Bornmann, L.; Mutz, R.; Daniel, H.-D.: Multilevel-statistical reformulation of citation-based university rankings : the Leiden ranking 2011/2012 (2013) 5.40
    5.3989544 = sum of:
      5.3989544 = sum of:
        1.3063266 = weight(author_txt:bornmann in 3008) [ClassicSimilarity], result of:
          1.3063266 = score(doc=3008,freq=1.0), product of:
            0.46289477 = queryWeight, product of:
              7.52555 = idf(docFreq=61, maxDocs=42306)
              0.06150976 = queryNorm
            2.822081 = fieldWeight in 3008, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              7.52555 = idf(docFreq=61, maxDocs=42306)
              0.375 = fieldNorm(doc=3008)
        1.6630177 = weight(author_txt:daniel in 3008) [ClassicSimilarity], result of:
          1.6630177 = score(doc=3008,freq=1.0), product of:
            0.54372466 = queryWeight, product of:
              1.083798 = boost
              8.156177 = idf(docFreq=32, maxDocs=42306)
              0.06150976 = queryNorm
            3.058566 = fieldWeight in 3008, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              8.156177 = idf(docFreq=32, maxDocs=42306)
              0.375 = fieldNorm(doc=3008)
        2.4296103 = weight(author_txt:mutz in 3008) [ClassicSimilarity], result of:
          2.4296103 = score(doc=3008,freq=1.0), product of:
            0.70006573 = queryWeight, product of:
              1.2297825 = boost
              9.254789 = idf(docFreq=10, maxDocs=42306)
              0.06150976 = queryNorm
            3.470546 = fieldWeight in 3008, product of:
              1.0 = tf(freq=1.0), with freq of:
                1.0 = termFreq=1.0
              9.254789 = idf(docFreq=10, maxDocs=42306)
              0.375 = fieldNorm(doc=3008)
    
  4. Bornmann, L.; Mutz, R.: From P100 to P100' : a new citation-rank approach (2014) 3.32
    3.320833 = sum of:
      3.320833 = product of:
        4.9812493 = sum of:
          1.7417688 = weight(author_txt:bornmann in 3432) [ClassicSimilarity], result of:
            1.7417688 = score(doc=3432,freq=1.0), product of:
              0.46289477 = queryWeight, product of:
                7.52555 = idf(docFreq=61, maxDocs=42306)
                0.06150976 = queryNorm
              3.762775 = fieldWeight in 3432, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.52555 = idf(docFreq=61, maxDocs=42306)
                0.5 = fieldNorm(doc=3432)
          3.2394805 = weight(author_txt:mutz in 3432) [ClassicSimilarity], result of:
            3.2394805 = score(doc=3432,freq=1.0), product of:
              0.70006573 = queryWeight, product of:
                1.2297825 = boost
                9.254789 = idf(docFreq=10, maxDocs=42306)
                0.06150976 = queryNorm
              4.6273947 = fieldWeight in 3432, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.254789 = idf(docFreq=10, maxDocs=42306)
                0.5 = fieldNorm(doc=3432)
        0.6666667 = coord(2/3)
    
  5. Bornmann, L.; Mutz, R.: Growth rates of modern science : a bibliometric analysis based on the number of publications and cited references (2015) 3.32
    3.320833 = sum of:
      3.320833 = product of:
        4.9812493 = sum of:
          1.7417688 = weight(author_txt:bornmann in 4262) [ClassicSimilarity], result of:
            1.7417688 = score(doc=4262,freq=1.0), product of:
              0.46289477 = queryWeight, product of:
                7.52555 = idf(docFreq=61, maxDocs=42306)
                0.06150976 = queryNorm
              3.762775 = fieldWeight in 4262, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                7.52555 = idf(docFreq=61, maxDocs=42306)
                0.5 = fieldNorm(doc=4262)
          3.2394805 = weight(author_txt:mutz in 4262) [ClassicSimilarity], result of:
            3.2394805 = score(doc=4262,freq=1.0), product of:
              0.70006573 = queryWeight, product of:
                1.2297825 = boost
                9.254789 = idf(docFreq=10, maxDocs=42306)
                0.06150976 = queryNorm
              4.6273947 = fieldWeight in 4262, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.254789 = idf(docFreq=10, maxDocs=42306)
                0.5 = fieldNorm(doc=4262)
        0.6666667 = coord(2/3)
    

Similar documents (content)

  1. Daniel, H.-D.: Guardians of science : fairness and reliabiblity of peer review (1993) 0.22
    0.22291785 = sum of:
      0.22291785 = product of:
        0.9288244 = sum of:
          0.04922397 = weight(abstract_txt:review in 3714) [ClassicSimilarity], result of:
            0.04922397 = score(doc=3714,freq=4.0), product of:
              0.06417956 = queryWeight, product of:
                1.0607752 = boost
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.012325754 = queryNorm
              0.76697266 = fieldWeight in 3714, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.078125 = fieldNorm(doc=3714)
          0.04148105 = weight(abstract_txt:decisions in 3714) [ClassicSimilarity], result of:
            0.04148105 = score(doc=3714,freq=1.0), product of:
              0.09089336 = queryWeight, product of:
                1.2623827 = boost
                5.841543 = idf(docFreq=333, maxDocs=42306)
                0.012325754 = queryNorm
              0.45637056 = fieldWeight in 3714, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.841543 = idf(docFreq=333, maxDocs=42306)
                0.078125 = fieldNorm(doc=3714)
          0.028319223 = weight(abstract_txt:evaluation in 3714) [ClassicSimilarity], result of:
            0.028319223 = score(doc=3714,freq=1.0), product of:
              0.08067091 = queryWeight, product of:
                1.4565623 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.012325754 = queryNorm
              0.3510463 = fieldWeight in 3714, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.078125 = fieldNorm(doc=3714)
          0.30306968 = weight(abstract_txt:peer in 3714) [ClassicSimilarity], result of:
            0.30306968 = score(doc=3714,freq=4.0), product of:
              0.29260874 = queryWeight, product of:
                3.581283 = boost
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.012325754 = queryNorm
              1.0357506 = fieldWeight in 3714, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.078125 = fieldNorm(doc=3714)
          0.1665216 = weight(abstract_txt:validity in 3714) [ClassicSimilarity], result of:
            0.1665216 = score(doc=3714,freq=1.0), product of:
              0.31159657 = queryWeight, product of:
                3.695654 = boost
                6.8405 = idf(docFreq=122, maxDocs=42306)
                0.012325754 = queryNorm
              0.53441405 = fieldWeight in 3714, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.8405 = idf(docFreq=122, maxDocs=42306)
                0.078125 = fieldNorm(doc=3714)
          0.3402089 = weight(abstract_txt:fairness in 3714) [ClassicSimilarity], result of:
            0.3402089 = score(doc=3714,freq=1.0), product of:
              0.46573558 = queryWeight, product of:
                4.041194 = boost
                9.3501 = idf(docFreq=9, maxDocs=42306)
                0.012325754 = queryNorm
              0.7304765 = fieldWeight in 3714, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                9.3501 = idf(docFreq=9, maxDocs=42306)
                0.078125 = fieldNorm(doc=3714)
        0.24 = coord(6/25)
    
  2. Lee, C.J.; Sugimoto, C.R.; Zhang, G.; Cronin, B.: Bias in peer review (2013) 0.17
    0.16781266 = sum of:
      0.16781266 = product of:
        0.8390633 = sum of:
          0.052093755 = weight(abstract_txt:review in 2526) [ClassicSimilarity], result of:
            0.052093755 = score(doc=2526,freq=7.0), product of:
              0.06417956 = queryWeight, product of:
                1.0607752 = boost
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.012325754 = queryNorm
              0.8116876 = fieldWeight in 2526, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.0625 = fieldNorm(doc=2526)
          0.031304065 = weight(abstract_txt:research in 2526) [ClassicSimilarity], result of:
            0.031304065 = score(doc=2526,freq=5.0), product of:
              0.06938988 = queryWeight, product of:
                1.7439861 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.012325754 = queryNorm
              0.451133 = fieldWeight in 2526, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0625 = fieldNorm(doc=2526)
          0.09316632 = weight(abstract_txt:funding in 2526) [ClassicSimilarity], result of:
            0.09316632 = score(doc=2526,freq=1.0), product of:
              0.22790557 = queryWeight, product of:
                2.8269455 = boost
                6.5406966 = idf(docFreq=165, maxDocs=42306)
                0.012325754 = queryNorm
              0.40879354 = fieldWeight in 2526, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.5406966 = idf(docFreq=165, maxDocs=42306)
                0.0625 = fieldNorm(doc=2526)
          0.36555272 = weight(abstract_txt:bias in 2526) [ClassicSimilarity], result of:
            0.36555272 = score(doc=2526,freq=9.0), product of:
              0.2725638 = queryWeight, product of:
                3.0915341 = boost
                7.1528745 = idf(docFreq=89, maxDocs=42306)
                0.012325754 = queryNorm
              1.341164 = fieldWeight in 2526, product of:
                3.0 = tf(freq=9.0), with freq of:
                  9.0 = termFreq=9.0
                7.1528745 = idf(docFreq=89, maxDocs=42306)
                0.0625 = fieldNorm(doc=2526)
          0.2969464 = weight(abstract_txt:peer in 2526) [ClassicSimilarity], result of:
            0.2969464 = score(doc=2526,freq=6.0), product of:
              0.29260874 = queryWeight, product of:
                3.581283 = boost
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.012325754 = queryNorm
              1.0148242 = fieldWeight in 2526, product of:
                2.4494898 = tf(freq=6.0), with freq of:
                  6.0 = termFreq=6.0
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.0625 = fieldNorm(doc=2526)
        0.2 = coord(5/25)
    
  3. Zhao, Y.W.; Chi. C.-H.; Heuvel, W.J. van den: Imperfect referees : reducing the impact of multiple biases in peer review (2015) 0.16
    0.16154292 = sum of:
      0.16154292 = product of:
        0.6730955 = sum of:
          0.039379176 = weight(abstract_txt:review in 4272) [ClassicSimilarity], result of:
            0.039379176 = score(doc=4272,freq=4.0), product of:
              0.06417956 = queryWeight, product of:
                1.0607752 = boost
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.012325754 = queryNorm
              0.61357814 = fieldWeight in 4272, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.0625 = fieldNorm(doc=4272)
          0.025441544 = weight(abstract_txt:real in 4272) [ClassicSimilarity], result of:
            0.025441544 = score(doc=4272,freq=1.0), product of:
              0.07613797 = queryWeight, product of:
                1.155382 = boost
                5.346409 = idf(docFreq=547, maxDocs=42306)
                0.012325754 = queryNorm
              0.33415055 = fieldWeight in 4272, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                5.346409 = idf(docFreq=547, maxDocs=42306)
                0.0625 = fieldNorm(doc=4272)
          0.11183543 = weight(abstract_txt:biases in 4272) [ClassicSimilarity], result of:
            0.11183543 = score(doc=4272,freq=2.0), product of:
              0.16216181 = queryWeight, product of:
                1.6861619 = boost
                7.8025365 = idf(docFreq=46, maxDocs=42306)
                0.012325754 = queryNorm
              0.6896533 = fieldWeight in 4272, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                7.8025365 = idf(docFreq=46, maxDocs=42306)
                0.0625 = fieldNorm(doc=4272)
          0.013999604 = weight(abstract_txt:research in 4272) [ClassicSimilarity], result of:
            0.013999604 = score(doc=4272,freq=1.0), product of:
              0.06938988 = queryWeight, product of:
                1.7439861 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.012325754 = queryNorm
              0.20175281 = fieldWeight in 4272, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0625 = fieldNorm(doc=4272)
          0.27246693 = weight(abstract_txt:bias in 4272) [ClassicSimilarity], result of:
            0.27246693 = score(doc=4272,freq=5.0), product of:
              0.2725638 = queryWeight, product of:
                3.0915341 = boost
                7.1528745 = idf(docFreq=89, maxDocs=42306)
                0.012325754 = queryNorm
              0.99964464 = fieldWeight in 4272, product of:
                2.236068 = tf(freq=5.0), with freq of:
                  5.0 = termFreq=5.0
                7.1528745 = idf(docFreq=89, maxDocs=42306)
                0.0625 = fieldNorm(doc=4272)
          0.20997281 = weight(abstract_txt:peer in 4272) [ClassicSimilarity], result of:
            0.20997281 = score(doc=4272,freq=3.0), product of:
              0.29260874 = queryWeight, product of:
                3.581283 = boost
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.012325754 = queryNorm
              0.717589 = fieldWeight in 4272, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.0625 = fieldNorm(doc=4272)
        0.24 = coord(6/25)
    
  4. Abramo, G.; D'Angelo, C.A.: ¬A decision support system for public research organizations participating in national research assessment exercises (2009) 0.15
    0.1540713 = sum of:
      0.1540713 = product of:
        0.48147282 = sum of:
          0.027845282 = weight(abstract_txt:review in 124) [ClassicSimilarity], result of:
            0.027845282 = score(doc=124,freq=2.0), product of:
              0.06417956 = queryWeight, product of:
                1.0607752 = boost
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.012325754 = queryNorm
              0.43386525 = fieldWeight in 124, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.035979774 = weight(abstract_txt:real in 124) [ClassicSimilarity], result of:
            0.035979774 = score(doc=124,freq=2.0), product of:
              0.07613797 = queryWeight, product of:
                1.155382 = boost
                5.346409 = idf(docFreq=547, maxDocs=42306)
                0.012325754 = queryNorm
              0.47256023 = fieldWeight in 124, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.346409 = idf(docFreq=547, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.01921093 = weight(abstract_txt:approach in 124) [ClassicSimilarity], result of:
            0.01921093 = score(doc=124,freq=2.0), product of:
              0.057362095 = queryWeight, product of:
                1.2282397 = boost
                3.789033 = idf(docFreq=2600, maxDocs=42306)
                0.012325754 = queryNorm
              0.33490637 = fieldWeight in 124, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.789033 = idf(docFreq=2600, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.045310758 = weight(abstract_txt:evaluation in 124) [ClassicSimilarity], result of:
            0.045310758 = score(doc=124,freq=4.0), product of:
              0.08067091 = queryWeight, product of:
                1.4565623 = boost
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.012325754 = queryNorm
              0.56167406 = fieldWeight in 124, product of:
                2.0 = tf(freq=4.0), with freq of:
                  4.0 = termFreq=4.0
                4.4933925 = idf(docFreq=1285, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.012887429 = weight(abstract_txt:data in 124) [ClassicSimilarity], result of:
            0.012887429 = score(doc=124,freq=1.0), product of:
              0.06095741 = queryWeight, product of:
                1.4620197 = boost
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.012325754 = queryNorm
              0.21141694 = fieldWeight in 124, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                3.382671 = idf(docFreq=3904, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.037039466 = weight(abstract_txt:research in 124) [ClassicSimilarity], result of:
            0.037039466 = score(doc=124,freq=7.0), product of:
              0.06938988 = queryWeight, product of:
                1.7439861 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.012325754 = queryNorm
              0.5337877 = fieldWeight in 124, product of:
                2.6457512 = tf(freq=7.0), with freq of:
                  7.0 = termFreq=7.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.13175708 = weight(abstract_txt:funding in 124) [ClassicSimilarity], result of:
            0.13175708 = score(doc=124,freq=2.0), product of:
              0.22790557 = queryWeight, product of:
                2.8269455 = boost
                6.5406966 = idf(docFreq=165, maxDocs=42306)
                0.012325754 = queryNorm
              0.57812136 = fieldWeight in 124, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.5406966 = idf(docFreq=165, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
          0.17144209 = weight(abstract_txt:peer in 124) [ClassicSimilarity], result of:
            0.17144209 = score(doc=124,freq=2.0), product of:
              0.29260874 = queryWeight, product of:
                3.581283 = boost
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.012325754 = queryNorm
              0.585909 = fieldWeight in 124, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.0625 = fieldNorm(doc=124)
        0.32 = coord(8/25)
    
  5. Bornmann, L.; Schier, H.; Marx, W.; Daniel, H.-D.: Is interactive open access publishing able to identify high-impact submissions? : a study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classes (2011) 0.14
    0.14457361 = sum of:
      0.14457361 = product of:
        0.60239005 = sum of:
          0.019689588 = weight(abstract_txt:review in 1133) [ClassicSimilarity], result of:
            0.019689588 = score(doc=1133,freq=1.0), product of:
              0.06417956 = queryWeight, product of:
                1.0607752 = boost
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.012325754 = queryNorm
              0.30678907 = fieldWeight in 1133, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                4.908625 = idf(docFreq=848, maxDocs=42306)
                0.0625 = fieldNorm(doc=1133)
          0.046930455 = weight(abstract_txt:decisions in 1133) [ClassicSimilarity], result of:
            0.046930455 = score(doc=1133,freq=2.0), product of:
              0.09089336 = queryWeight, product of:
                1.2623827 = boost
                5.841543 = idf(docFreq=333, maxDocs=42306)
                0.012325754 = queryNorm
              0.51632434 = fieldWeight in 1133, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                5.841543 = idf(docFreq=333, maxDocs=42306)
                0.0625 = fieldNorm(doc=1133)
          0.01979843 = weight(abstract_txt:research in 1133) [ClassicSimilarity], result of:
            0.01979843 = score(doc=1133,freq=2.0), product of:
              0.06938988 = queryWeight, product of:
                1.7439861 = boost
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.012325754 = queryNorm
              0.28532156 = fieldWeight in 1133, product of:
                1.4142135 = tf(freq=2.0), with freq of:
                  2.0 = termFreq=2.0
                3.228045 = idf(docFreq=4557, maxDocs=42306)
                0.0625 = fieldNorm(doc=1133)
          0.16400464 = weight(abstract_txt:predictive in 1133) [ClassicSimilarity], result of:
            0.16400464 = score(doc=1133,freq=3.0), product of:
              0.18285324 = queryWeight, product of:
                1.7905082 = boost
                8.285388 = idf(docFreq=28, maxDocs=42306)
                0.012325754 = queryNorm
              0.89691955 = fieldWeight in 1133, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                8.285388 = idf(docFreq=28, maxDocs=42306)
                0.0625 = fieldNorm(doc=1133)
          0.12122787 = weight(abstract_txt:peer in 1133) [ClassicSimilarity], result of:
            0.12122787 = score(doc=1133,freq=1.0), product of:
              0.29260874 = queryWeight, product of:
                3.581283 = boost
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.012325754 = queryNorm
              0.41430023 = fieldWeight in 1133, product of:
                1.0 = tf(freq=1.0), with freq of:
                  1.0 = termFreq=1.0
                6.6288037 = idf(docFreq=151, maxDocs=42306)
                0.0625 = fieldNorm(doc=1133)
          0.23073907 = weight(abstract_txt:validity in 1133) [ClassicSimilarity], result of:
            0.23073907 = score(doc=1133,freq=3.0), product of:
              0.31159657 = queryWeight, product of:
                3.695654 = boost
                6.8405 = idf(docFreq=122, maxDocs=42306)
                0.012325754 = queryNorm
              0.7405058 = fieldWeight in 1133, product of:
                1.7320508 = tf(freq=3.0), with freq of:
                  3.0 = termFreq=3.0
                6.8405 = idf(docFreq=122, maxDocs=42306)
                0.0625 = fieldNorm(doc=1133)
        0.24 = coord(6/25)