Sun, M.; Danfa, J.B.; Teplitskiy, M.: Does double-blind peer review reduce bias? : evidence from a top computer science conference (2022)
0.03
0.025427736 = sum of:
0.021531772 = product of:
0.08612709 = sum of:
0.08612709 = weight(_text_:authors in 562) [ClassicSimilarity], result of:
0.08612709 = score(doc=562,freq=4.0), product of:
0.24182312 = queryWeight, product of:
4.558814 = idf(docFreq=1258, maxDocs=44218)
0.053045183 = queryNorm
0.35615736 = fieldWeight in 562, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
4.558814 = idf(docFreq=1258, maxDocs=44218)
0.0390625 = fieldNorm(doc=562)
0.25 = coord(1/4)
0.0038959642 = product of:
0.0077919285 = sum of:
0.0077919285 = weight(_text_:a in 562) [ClassicSimilarity], result of:
0.0077919285 = score(doc=562,freq=8.0), product of:
0.06116359 = queryWeight, product of:
1.153047 = idf(docFreq=37942, maxDocs=44218)
0.053045183 = queryNorm
0.12739488 = fieldWeight in 562, product of:
2.828427 = tf(freq=8.0), with freq of:
8.0 = termFreq=8.0
1.153047 = idf(docFreq=37942, maxDocs=44218)
0.0390625 = fieldNorm(doc=562)
0.5 = coord(1/2)
- Abstract
- Peer review is essential for advancing scientific research, but there are long-standing concerns that authors' prestige or other characteristics can bias reviewers. Double-blind peer review has been proposed as a way to reduce reviewer bias, but the evidence for its effectiveness is limited and mixed. Here, we examine the effects of double-blind peer review by analyzing the review files of 5,027 papers submitted to a top computer science conference that changed its reviewing format from single- to double-blind in 2018. First, we find that the scores given to the most prestigious authors significantly decreased after switching to double-blind review. However, because many of these papers were above the threshold for acceptance, the change did not affect paper acceptance significantly. Second, the inter-reviewer disagreement increased significantly in the double-blind format. Third, papers rejected in the single-blind format are cited more than those rejected under double-blind, suggesting that double-blind review better excludes poorer quality papers. Lastly, an apparently unrelated change in the rating scale from 10 to 4 points likely reduced prestige bias significantly such that papers' acceptance was affected. These results support the effectiveness of double-blind review in reducing biases, while opening new research directions on the impact of peer-review formats.
- Type
- a