Sun, M.; Danfa, J.B.; Teplitskiy, M.: Does double-blind peer review reduce bias? : evidence from a top computer science conference (2022)
0.01
0.007723937 = product of:
0.04634362 = sum of:
0.04634362 = weight(_text_:computer in 562) [ClassicSimilarity], result of:
0.04634362 = score(doc=562,freq=4.0), product of:
0.16231956 = queryWeight, product of:
3.6545093 = idf(docFreq=3109, maxDocs=44218)
0.044416238 = queryNorm
0.28550854 = fieldWeight in 562, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
3.6545093 = idf(docFreq=3109, maxDocs=44218)
0.0390625 = fieldNorm(doc=562)
0.16666667 = coord(1/6)
- Abstract
- Peer review is essential for advancing scientific research, but there are long-standing concerns that authors' prestige or other characteristics can bias reviewers. Double-blind peer review has been proposed as a way to reduce reviewer bias, but the evidence for its effectiveness is limited and mixed. Here, we examine the effects of double-blind peer review by analyzing the review files of 5,027 papers submitted to a top computer science conference that changed its reviewing format from single- to double-blind in 2018. First, we find that the scores given to the most prestigious authors significantly decreased after switching to double-blind review. However, because many of these papers were above the threshold for acceptance, the change did not affect paper acceptance significantly. Second, the inter-reviewer disagreement increased significantly in the double-blind format. Third, papers rejected in the single-blind format are cited more than those rejected under double-blind, suggesting that double-blind review better excludes poorer quality papers. Lastly, an apparently unrelated change in the rating scale from 10 to 4 points likely reduced prestige bias significantly such that papers' acceptance was affected. These results support the effectiveness of double-blind review in reducing biases, while opening new research directions on the impact of peer-review formats.
