The advent of artificial intelligence (AI) has sparked interest in its potential use to enhance and streamline the peer review process, a crucial aspect of scholarly research. Here we provide a high level summary of some of the arguments surrounding the use of AI in peer review and explore how it can impact the quality, efficiency, and fairness of the process and therefore scientific integrity and quality. Below we provide some insightful reviews that have been published on the topic and were used for this summary.
AI-assisted peer review systems have the potential to enhance review quality and efficiency. They can be a valuable tool for flagging potentially low-quality or controversial studies. By automating certain steps of the review process, these systems can help reviewers focus on more critical aspects, ultimately leading to improved feedback and decision-making. Furthermore, they can provide a tool for complementary decision support for editors, reviewers and authors. Its ability to analyze vast amounts of data can speed up the overall review time while maintaining rigorous evaluation standards. The systematic integration of AI in the peer review process could lead to the creation of a standardized referenceable peer review database. This would not only improve transparency but also provide valuable insights for future research and evaluations. Finally, AI platforms have shown varying success in evidence synthesis, which can significantly impact the quality and reliability of published research.
Despite many of these potential benefits, the use of AI in peer review harbors many major problems. For example, it could and certainly will, if not carefully monitored introduce biases against authors from non-English speaking countries and economically disadvantaged regions. Ensuring that AI-developed models are not overfitting to training data is crucial to achieving a more inclusive and equitable peer review process. The use of AI in peer review also raises important ethical concerns that must be addressed at multiple levels. For example, it has been pointed out that the use of AI in grant peer review presents a breach of confidentiality (Using AI in Peer Review Is a Breach of Confidentiality – NIH Extramural Nexus) and several granting agencies have called for a complete ban on AI use in the peer review and grant writing process (Science funding agencies say no to using AI for peer review | Science | AAAS). Ensuring that AI algorithms are transparent, unbiased, and do not compromise privacy and confidentiality is essential for maintaining the integrity of the process.
Peer review remains the backbone of scholarly research, and the integration of AI has the potential to revolutionize this process positively. By leveraging AI to enhance review quality, efficiency, and transparency, the scientific community can strengthen its commitment to producing reliable and impactful research. However, it is crucial to approach the use of AI in peer review with caution, addressing potential biases, ethical concerns, and limitations to ensure a fair and inclusive system. As AI continues to advance, its responsible integration can reshape peer review for the better, empowering researchers and reviewers to navigate the ever-changing scientific landscape with confidence and accuracy.
References:
1. Checco, A., et al., AI-assisted peer review. Humanities and Social Sciences Communications, 2021. 8: p. 1-11.
2. Ghosal, T., et al., Can your paper evade the editors axe? Towards an AI assisted peer review system. arXiv: Digital Libraries, 2018.
3. Heaven, D., AI peer reviewers unleashed to ease publishing grind. Nature, 2018. 563: p. 609 – 610.
4. Shaw, J., Emerging Paradigms for Ethical Review of Research Using Artificial Intelligence. The American Journal of Bioethics, 2022. 22: p. 42 – 44.
5. Thelwall, M.A. Artificial Intelligence, Automation and Peer Review. 2019.