Search (38 results, page 1 of 2)

0.04838429 = product of:
  0.07257643 = sum of:
    0.032007344 = weight(_text_:on in 1239) [ClassicSimilarity], result of:
      0.032007344 = score(doc=1239,freq=2.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.29160398 = fieldWeight in 1239, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.09375 = fieldNorm(doc=1239)
    0.040569093 = product of:
      0.081138186 = sum of:
        0.081138186 = weight(_text_:22 in 1239) [ClassicSimilarity], result of:
          0.081138186 = score(doc=1239,freq=2.0), product of:
            0.1747608 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.04990557 = queryNorm
            0.46428138 = fieldWeight in 1239, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.09375 = fieldNorm(doc=1239)
      0.5 = coord(1/2)
  0.6666667 = coord(2/3)

Date

18. 3.2014 19:13:22

0.032256197 = product of:
  0.048384294 = sum of:
    0.021338228 = weight(_text_:on in 1431) [ClassicSimilarity], result of:
      0.021338228 = score(doc=1431,freq=2.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.19440265 = fieldWeight in 1431, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0625 = fieldNorm(doc=1431)
    0.027046064 = product of:
      0.054092128 = sum of:
        0.054092128 = weight(_text_:22 in 1431) [ClassicSimilarity], result of:
          0.054092128 = score(doc=1431,freq=2.0), product of:
            0.1747608 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.04990557 = queryNorm
            0.30952093 = fieldWeight in 1431, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.0625 = fieldNorm(doc=1431)
      0.5 = coord(1/2)
  0.6666667 = coord(2/3)

Abstract

Properties of a percentile-based rating scale needed in bibliometrics are formulated. Based on these properties, P100 was recently introduced as a new citation-rank approach (Bornmann, Leydesdorff, & Wang, 2013). In this paper, we conceptualize P100 and propose an improvement which we call P100'. Advantages and disadvantages of citation-rank indicators are noted.

Date

22. 8.2014 17:05:18

0.032002483 = product of:
  0.048003722 = sum of:
    0.027719175 = weight(_text_:on in 656) [ClassicSimilarity], result of:
      0.027719175 = score(doc=656,freq=6.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.25253648 = fieldWeight in 656, product of:
          2.4494898 = tf(freq=6.0), with freq of:
            6.0 = termFreq=6.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=656)
    0.020284547 = product of:
      0.040569093 = sum of:
        0.040569093 = weight(_text_:22 in 656) [ClassicSimilarity], result of:
          0.040569093 = score(doc=656,freq=2.0), product of:
            0.1747608 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.04990557 = queryNorm
            0.23214069 = fieldWeight in 656, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.046875 = fieldNorm(doc=656)
      0.5 = coord(1/2)
  0.6666667 = coord(2/3)

Abstract

According to current research in bibliometrics, percentiles (or percentile rank classes) are the most suitable method for normalizing the citation counts of individual publications in terms of the subject area, the document type, and the publication year. Up to now, bibliometric research has concerned itself primarily with the calculation of percentiles. This study suggests how percentiles (and percentile rank classes) can be analyzed meaningfully for an evaluation study. Publication sets from four universities are compared with each other to provide sample data. These suggestions take into account on the one hand the distribution of percentiles over the publications in the sets (universities here) and on the other hand concentrate on the range of publications with the highest citation impact-that is, the range that is usually of most interest in the evaluation of scientific performance.

Date

22. 3.2013 19:44:17

0.028611436 = product of:
  0.042917155 = sum of:
    0.02263261 = weight(_text_:on in 4681) [ClassicSimilarity], result of:
      0.02263261 = score(doc=4681,freq=4.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.20619515 = fieldWeight in 4681, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=4681)
    0.020284547 = product of:
      0.040569093 = sum of:
        0.040569093 = weight(_text_:22 in 4681) [ClassicSimilarity], result of:
          0.040569093 = score(doc=4681,freq=2.0), product of:
            0.1747608 = queryWeight, product of:
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.04990557 = queryNorm
            0.23214069 = fieldWeight in 4681, product of:
              1.4142135 = tf(freq=2.0), with freq of:
                2.0 = termFreq=2.0
              3.5018296 = idf(docFreq=3622, maxDocs=44218)
              0.046875 = fieldNorm(doc=4681)
      0.5 = coord(1/2)
  0.6666667 = coord(2/3)

Abstract

A recent publication in Nature reports that public R&D funding is only weakly correlated with the citation impact of a nation's articles as measured by the field-weighted citation index (FWCI; defined by Scopus). On the basis of the supplementary data, we up-scaled the design using Web of Science data for the decade 2003-2013 and OECD funding data for the corresponding decade assuming a 2-year delay (2001-2011). Using negative binomial regression analysis, we found very small coefficients, but the effects of international collaboration are positive and statistically significant, whereas the effects of government funding are negative, an order of magnitude smaller, and statistically nonsignificant (in two of three analyses). In other words, international collaboration improves the impact of research articles, whereas more government funding tends to have a small adverse effect when comparing OECD countries.

Date

8. 1.2019 18:22:45

0.0124473 = product of:
  0.0373419 = sum of:
    0.0373419 = weight(_text_:on in 1238) [ClassicSimilarity], result of:
      0.0373419 = score(doc=1238,freq=8.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.34020463 = fieldWeight in 1238, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0546875 = fieldNorm(doc=1238)
  0.33333334 = coord(1/3)

Abstract

We introduce the quantitative method named "Reference Publication Year Spectroscopy" (RPYS). With this method one can determine the historical roots of research fields and quantify their impact on current research. RPYS is based on the analysis of the frequency with which references are cited in the publications of a specific research field in terms of the publication years of these cited references. The origins show up in the form of more or less pronounced peaks mostly caused by individual publications that are cited particularly frequently. In this study, we use research on graphene and on solar cells to illustrate how RPYS functions, and what results it can deliver.

0.0124473 = product of:
  0.0373419 = sum of:
    0.0373419 = weight(_text_:on in 1247) [ClassicSimilarity], result of:
      0.0373419 = score(doc=1247,freq=2.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.34020463 = fieldWeight in 1247, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.109375 = fieldNorm(doc=1247)
  0.33333334 = coord(1/3)

0.011761595 = product of:
  0.035284784 = sum of:
    0.035284784 = weight(_text_:on in 3095) [ClassicSimilarity], result of:
      0.035284784 = score(doc=3095,freq=14.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.3214632 = fieldWeight in 3095, product of:
          3.7416575 = tf(freq=14.0), with freq of:
            14.0 = termFreq=14.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0390625 = fieldNorm(doc=3095)
  0.33333334 = coord(1/3)

Abstract

Citation data for institutes are generally provided as numbers of citations or as relative citation rates (as, for example, in the Leiden Ranking). These numbers can then be compared between the institutes. This study aims to present a new approach for the evaluation of citation data at the institutional level, based on regression models. As example data, the study includes all articles and reviews from the Web of Science for the publication year 2003 (n?=?886,416 papers). The study is based on an in-house database of the Max Planck Society. The study investigates how much the expected number of citations for a publication changes if it contains the address of an institute. The calculation of the expected values allows, on the one hand, investigating how the citation impact of the papers of an institute appears in comparison with the total of all papers. On the other hand, the expected values for several institutes can be compared with one another or with a set of randomly selected publications. Besides the institutes, the regression models include factors which can be assumed to have a general influence on citation counts (e.g., the number of authors).

0.010669115 = product of:
  0.032007344 = sum of:
    0.032007344 = weight(_text_:on in 4485) [ClassicSimilarity], result of:
      0.032007344 = score(doc=4485,freq=8.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.29160398 = fieldWeight in 4485, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=4485)
  0.33333334 = coord(1/3)

Abstract

We submit newly developed citation impact indicators based not on arithmetic averages of citations but on percentile ranks. Citation distributions are-as a rule-highly skewed and should not be arithmetically averaged. With percentile ranks, the citation score of each paper is rated in terms of its percentile in the citation distribution. The percentile ranks approach allows for the formulation of a more abstract indicator scheme that can be used to organize and/or schematize different impact indicators according to three degrees of freedom: the selection of the reference sets, the evaluation criteria, and the choice of whether or not to define the publication sets as independent. Bibliometric data of seven principal investigators (PIs) of the Academic Medical Center of the University of Amsterdam are used as an exemplary dataset. We demonstrate that the proposed family indicators [R(6), R(100), R(6, k), R(100, k)] are an improvement on averages-based indicators because one can account for the shape of the distributions of citations over papers.

0.010669115 = product of:
  0.032007344 = sum of:
    0.032007344 = weight(_text_:on in 752) [ClassicSimilarity], result of:
      0.032007344 = score(doc=752,freq=2.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.29160398 = fieldWeight in 752, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.09375 = fieldNorm(doc=752)
  0.33333334 = coord(1/3)

0.010669115 = product of:
  0.032007344 = sum of:
    0.032007344 = weight(_text_:on in 1188) [ClassicSimilarity], result of:
      0.032007344 = score(doc=1188,freq=2.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.29160398 = fieldWeight in 1188, product of:
          1.4142135 = tf(freq=2.0), with freq of:
            2.0 = termFreq=2.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.09375 = fieldNorm(doc=1188)
  0.33333334 = coord(1/3)

0.009239726 = product of:
  0.027719175 = sum of:
    0.027719175 = weight(_text_:on in 1109) [ClassicSimilarity], result of:
      0.027719175 = score(doc=1109,freq=6.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.25253648 = fieldWeight in 1109, product of:
          2.4494898 = tf(freq=6.0), with freq of:
            6.0 = termFreq=6.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=1109)
  0.33333334 = coord(1/3)

Abstract

Using data compiled for the SCImago Institutions Ranking, we look at whether the subject area type an institution (university or research-focused institution) belongs to (in terms of the fields researched) has an influence on its ranking position. We used latent class analysis to categorize institutions based on their publications in certain subject areas. Even though this categorization does not relate directly to scientific performance, our results show that it exercises an important influence on the outcome of a performance measurement: Certain subject area types of institutions have an advantage in the ranking positions when compared with others. This advantage manifests itself not only when performance is measured with an indicator that is not field-normalized but also for indicators that are field-normalized.

0.008890929 = product of:
  0.026672786 = sum of:
    0.026672786 = weight(_text_:on in 3539) [ClassicSimilarity], result of:
      0.026672786 = score(doc=3539,freq=8.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.24300331 = fieldWeight in 3539, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0390625 = fieldNorm(doc=3539)
  0.33333334 = coord(1/3)

Abstract

In recent years, the relationship of collaboration among scientists and the citation impact of papers have been frequently investigated. Most of the studies show that the two variables are closely related: An increasing collaboration activity (measured in terms of number of authors, number of affiliations, and number of countries) is associated with an increased citation impact. However, it is not clear whether the increased citation impact is based on the higher quality of papers that profit from more than one scientist giving expert input or other (citation-specific) factors. Thus, the current study addresses this question by using two comprehensive data sets with publications (in the biomedical area) including quality assessments by experts (F1000Prime member scores) and citation data for the publications. The study is based on more than 15,000 papers. Robust regression models are used to investigate the relationship between number of authors, number of affiliations, and number of countries, respectively, and citation impact-controlling for the papers' quality (measured by F1000Prime expert ratings). The results point out that the effect of collaboration activities on impact is largely independent of the papers' quality. The citation advantage is apparently not quality related; citation-specific factors (e.g., self-citations) seem to be important here.

0.008890929 = product of:
  0.026672786 = sum of:
    0.026672786 = weight(_text_:on in 4196) [ClassicSimilarity], result of:
      0.026672786 = score(doc=4196,freq=8.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.24300331 = fieldWeight in 4196, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0390625 = fieldNorm(doc=4196)
  0.33333334 = coord(1/3)

Abstract

The purpose of this paper is to propose an approach for identifying landmark papers in the long run. These publications reach a very high level of citation impact and are able to remain on this level across many citing years. In recent years, several studies have been published which deal with the citation history of publications and try to identify landmark publications. Design/methodology/approach In contrast to other studies published hitherto, this study is based on a broad data set with papers published between 1980 and 1990 for identifying the landmark papers. The authors analyzed the citation histories of about five million papers across 25 years. Findings The results of this study reveal that 1,013 papers (less than 0.02 percent) are "outstandingly cited" in the long run. The cluster analyses of the papers show that they received the high impact level very soon after publication and remained on this level over decades. Only a slight impact decline is visible over the years. Originality/value For practical reasons, approaches for identifying landmark papers should be as simple as possible. The approach proposed in this study is based on standard methods in bibliometrics.

0.008890929 = product of:
  0.026672786 = sum of:
    0.026672786 = weight(_text_:on in 5225) [ClassicSimilarity], result of:
      0.026672786 = score(doc=5225,freq=8.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.24300331 = fieldWeight in 5225, product of:
          2.828427 = tf(freq=8.0), with freq of:
            8.0 = termFreq=8.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0390625 = fieldNorm(doc=5225)
  0.33333334 = coord(1/3)

Abstract

The Leiden Rankings can be used for grouping research universities by considering universities which are not statistically significantly different as homogeneous sets. The groups and intergroup relations can be analyzed and visualized using tools from network analysis. Using the so-called "excellence indicator" PPtop-10%-the proportion of the top-10% most-highly-cited papers assigned to a university-we pursue a classification using (a) overlapping stability intervals, (b) statistical-significance tests, and (c) effect sizes of differences among 902 universities in 54 countries; we focus on the UK, Germany, Brazil, and the USA as national examples. Although the groupings remain largely the same using different statistical significance levels or overlapping stability intervals, these classifications are uncorrelated with those based on effect sizes. Effect sizes for the differences between universities are small (w < .2). The more detailed analysis of universities at the country level suggests that distinctions beyond three or perhaps four groups of universities (high, middle, low) may not be meaningful. Given similar institutional incentives, isomorphism within each eco-system of universities should not be underestimated. Our results suggest that networks based on overlapping stability intervals can provide a first impression of the relevant groupings among universities. However, the clusters are not well-defined divisions between groups of universities.

0.008801571 = product of:
  0.026404712 = sum of:
    0.026404712 = weight(_text_:on in 1293) [ClassicSimilarity], result of:
      0.026404712 = score(doc=1293,freq=4.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.24056101 = fieldWeight in 1293, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0546875 = fieldNorm(doc=1293)
  0.33333334 = coord(1/3)

Abstract

This Brief Communication discusses the benefits of citation analysis in research evaluation based on Galton's "Wisdom of Crowds" (1907). Citations are based on the assessment of many which is why they can be considered to have some credibility. However, we show that citations are incomplete assessments and that one cannot assume that a high number of citations correlates with a high level of usefulness. Only when one knows that a rarely cited paper has been widely read is it possible to say-strictly speaking-that it was obviously of little use for further research. Using a comparison with "like" data, we try to determine that cited reference analysis allows for a more meaningful analysis of bibliometric data than times-cited analysis.

0.0076997704 = product of:
  0.02309931 = sum of:
    0.02309931 = weight(_text_:on in 4132) [ClassicSimilarity], result of:
      0.02309931 = score(doc=4132,freq=6.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.21044704 = fieldWeight in 4132, product of:
          2.4494898 = tf(freq=6.0), with freq of:
            6.0 = termFreq=6.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0390625 = fieldNorm(doc=4132)
  0.33333334 = coord(1/3)

Abstract

In a comprehensive research project, we investigated the predictive validity of selection decisions and reviewers' ratings at the open access journal Atmospheric Chemistry and Physics (ACP). ACP is a high-impact journal publishing papers on the Earth's atmosphere and the underlying chemical and physical processes. Scientific journals have to deal with the following question concerning the predictive validity: Are in fact the "best" scientific works selected from the manuscripts submitted? In this study we examined whether selecting the "best" manuscripts means selecting papers that after publication show top citation performance as compared to other papers in this research area. First, we appraised the citation impact of later published manuscripts based on the percentile citedness rank classes of the population distribution (scaling in a specific subfield). Second, we analyzed the association between the decisions (n = 677 accepted or rejected, but published elsewhere manuscripts) or ratings (reviewers' ratings for n = 315 manuscripts), respectively, and the citation impact classes of the manuscripts. The results confirm the predictive validity of the ACP peer review system.

0.0076997704 = product of:
  0.02309931 = sum of:
    0.02309931 = weight(_text_:on in 1235) [ClassicSimilarity], result of:
      0.02309931 = score(doc=1235,freq=6.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.21044704 = fieldWeight in 1235, product of:
          2.4494898 = tf(freq=6.0), with freq of:
            6.0 = termFreq=6.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.0390625 = fieldNorm(doc=1235)
  0.33333334 = coord(1/3)

Abstract

University rankings generally present users with the problem of placing the results given for an institution in context. Only a comparison with the performance of all other institutions makes it possible to say exactly where an institution stands. In order to interpret the results of the SCImago Institutions Ranking (based on Scopus data) and the Leiden Ranking (based on Web of Science data), in this study we offer thresholds with which it is possible to assess whether an institution belongs to the top 1%, top 5%, top 10%, top 25%, or top 50% of institutions in the world. The thresholds are based on the excellence rate or PPtop 10%. Both indicators measure the proportion of an institution's publications which belong to the 10% most frequently cited publications and are the most important indicators for measuring institutional impact. For example, while an institution must achieve a value of 24.63% in the Leiden Ranking 2013 to be considered one of the top 1% of institutions worldwide, the SCImago Institutions Ranking requires 30.2%.

0.0075442037 = product of:
  0.02263261 = sum of:
    0.02263261 = weight(_text_:on in 4767) [ClassicSimilarity], result of:
      0.02263261 = score(doc=4767,freq=4.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.20619515 = fieldWeight in 4767, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=4767)
  0.33333334 = coord(1/3)

Abstract

The methods presented in this paper allow for a statistical analysis revealing centers of excellence around the world using programs that are freely available. Based on Web of Science data (a fee-based database), field-specific excellence can be identified in cities where highly cited papers were published more frequently than can be expected. Compared to the mapping approaches published hitherto, our approach is more analytically oriented by allowing the assessment of an observed number of excellent papers for a city against the expected number. Top performers in output are cities in which authors are located who publish a statistically significant higher number of highly cited papers than can be expected for these cities. As sample data for physics, chemistry, and psychology show, these cities do not necessarily have a high output of highly cited papers.

0.0075442037 = product of:
  0.02263261 = sum of:
    0.02263261 = weight(_text_:on in 2328) [ClassicSimilarity], result of:
      0.02263261 = score(doc=2328,freq=4.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.20619515 = fieldWeight in 2328, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=2328)
  0.33333334 = coord(1/3)

Abstract

Peer review is the backbone of modern science. F1000Prime is a postpublication peer review system of the biomedical literature (papers from medical and biological journals). This study is concerned with the interrater reliability and convergent validity of the peer recommendations formulated in the F1000Prime peer review system. The study is based on about 100,000 papers with recommendations from faculty members. Even if intersubjectivity plays a fundamental role in science, the analyses of the reliability of the F1000Prime peer review system show a rather low level of agreement between faculty members. This result is in agreement with most other studies that have been published on the journal peer review system. Logistic regression models are used to investigate the convergent validity of the F1000Prime peer review system. As the results show, the proportion of highly cited papers among those selected by the faculty members is significantly higher than expected. In addition, better recommendation scores are also associated with higher performing papers.

0.0075442037 = product of:
  0.02263261 = sum of:
    0.02263261 = weight(_text_:on in 2779) [ClassicSimilarity], result of:
      0.02263261 = score(doc=2779,freq=4.0), product of:
        0.109763056 = queryWeight, product of:
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.04990557 = queryNorm
        0.20619515 = fieldWeight in 2779, product of:
          2.0 = tf(freq=4.0), with freq of:
            4.0 = termFreq=4.0
          2.199415 = idf(docFreq=13325, maxDocs=44218)
          0.046875 = fieldNorm(doc=2779)
  0.33333334 = coord(1/3)

Abstract

Normalization of citation scores using reference sets based on Web of Science subject categories (WCs) has become an established ("best") practice in evaluative bibliometrics. For example, the Times Higher Education World University Rankings are, among other things, based on this operationalization. However, WCs were developed decades ago for the purpose of information retrieval and evolved incrementally with the database; the classification is machine-based and partially manually corrected. Using the WC "information science & library science" and the WCs attributed to journals in the field of "science and technology studies," we show that WCs do not provide sufficient analytical clarity to carry bibliometric normalization in evaluation practices because of "indexer effects." Can the compliance with "best practices" be replaced with an ambition to develop "best possible practices"? New research questions can then be envisaged.